2024-05-08T21:31:58.693 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T21:31:58.694 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067 --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} --owner scheduled_pdonnell@teuthology --verbose -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/orig.config.yaml 2024-05-08T21:31:58.754 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T21:31:58.848 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698067' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJVX97mDJEXNaOqRdr5Rb8RY/v65Psar9621oHycnyG5vaHwO+cMCD0Jt/Z2lq2UdZ5ZahueoM4n+thkGgdsKWw= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpw1gch/6cfIHOHGqcqq/gPtJsD37H8gxxpK4uHgBo+yEy7H2RQYwjoGQm291qoMG0z1zvRBB4pN0mGZe1eoOI= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJoabSVuC3EMgJgKpwxQjdeyV6SfBpQ4Q67vrOaPFQGVIQTnju/i7OJx3PqAX8GlS7fE7vm9Fs/l+RnGbliX3Ks= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T21:31:58.848 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T21:31:58.849 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T21:31:58.849 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T21:31:58.851 INFO:teuthology.task.internal:Checking packages... 2024-05-08T21:31:58.875 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T21:31:58.875 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T21:31:58.875 INFO:teuthology.packaging:ref: None 2024-05-08T21:31:58.875 INFO:teuthology.packaging:tag: None 2024-05-08T21:31:58.875 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T21:31:58.875 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:31:58.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T21:31:59.020 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T21:31:59.022 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T21:31:59.028 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T21:31:59.028 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T21:31:59.033 INFO:teuthology.task.internal:Saving configuration 2024-05-08T21:31:59.056 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T21:31:59.062 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T21:31:59.086 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 21:26:32.082391', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJVX97mDJEXNaOqRdr5Rb8RY/v65Psar9621oHycnyG5vaHwO+cMCD0Jt/Z2lq2UdZ5ZahueoM4n+thkGgdsKWw='} 2024-05-08T21:31:59.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 21:26:32.081300', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpw1gch/6cfIHOHGqcqq/gPtJsD37H8gxxpK4uHgBo+yEy7H2RQYwjoGQm291qoMG0z1zvRBB4pN0mGZe1eoOI='} 2024-05-08T21:31:59.130 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 21:26:32.080002', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJoabSVuC3EMgJgKpwxQjdeyV6SfBpQ4Q67vrOaPFQGVIQTnju/i7OJx3PqAX8GlS7fE7vm9Fs/l+RnGbliX3Ks='} 2024-05-08T21:31:59.130 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T21:31:59.136 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T21:31:59.136 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T21:31:59.136 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T21:31:59.136 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T21:31:59.268 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d57cfe160>, signals=[15]) 2024-05-08T21:31:59.268 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T21:31:59.275 INFO:teuthology.task.internal:Opening connections... 2024-05-08T21:31:59.275 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:31:59.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:31:59.396 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-05-08T21:31:59.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:31:59.468 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-05-08T21:31:59.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:31:59.552 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T21:31:59.559 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-05-08T21:31:59.580 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-05-08T21:31:59.580 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-05-08T21:31:59.640 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T21:31:59.640 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T21:31:59.640 INFO:teuthology.orchestra.run.smithi064.stdout:ID="rhel" 2024-05-08T21:31:59.640 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="fedora" 2024-05-08T21:31:59.640 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8.6" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T21:31:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T21:31:59.642 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-05-08T21:31:59.663 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-05-08T21:31:59.683 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-05-08T21:31:59.684 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-05-08T21:31:59.744 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T21:31:59.744 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T21:31:59.744 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel" 2024-05-08T21:31:59.744 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora" 2024-05-08T21:31:59.744 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.6" 2024-05-08T21:31:59.744 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T21:31:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T21:31:59.746 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-05-08T21:31:59.770 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-05-08T21:31:59.791 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-05-08T21:31:59.792 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T21:31:59.852 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T21:31:59.853 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T21:31:59.853 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-05-08T21:31:59.870 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T21:31:59.881 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T21:31:59.887 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T21:31:59.887 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-05-08T21:31:59.890 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-05-08T21:31:59.893 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-05-08T21:31:59.914 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T21:31:59.921 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T21:31:59.921 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-05-08T21:31:59.951 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-05-08T21:31:59.954 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-05-08T21:31:59.980 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T21:32:00.061 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T21:32:00.084 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T21:32:00.085 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T21:32:00.085 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T21:32:00.085 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T21:32:00.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:32:00.107 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-05-08T21:32:00.168 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T21:32:00.168 INFO:teuthology.task.kernel:Running kernel on smithi064: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T21:32:00.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-05-08T21:32:01.680 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:32:01.680 INFO:teuthology.orchestra.run.smithi064.stdout:Unable to read consumer identity 2024-05-08T21:32:01.941 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:32:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T21:32:01.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:32:03.642 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T21:32:05.212 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-05-08T21:32:09.035 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 457 kB/s | 24 kB 00:00 2024-05-08T21:32:09.837 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T21:32:09.837 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T21:32:09.919 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-08T21:32:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-05-08T21:32:09.921 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-08T21:32:10.050 DEBUG:teuthology.orchestra.run.smithi064:> echo no | sudo yum reinstall kernel || true 2024-05-08T21:32:10.406 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:32:10.406 INFO:teuthology.orchestra.run.smithi064.stdout:Unable to read consumer identity 2024-05-08T21:32:10.611 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:32:10.611 INFO:teuthology.orchestra.run.smithi064.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T21:32:10.611 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:32:10.937 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 09:32:08 PM UTC. 2024-05-08T21:32:11.017 INFO:teuthology.orchestra.run.smithi064.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T21:32:11.017 INFO:teuthology.orchestra.run.smithi064.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T21:32:11.068 INFO:teuthology.orchestra.run.smithi064.stderr:Error: No packages marked for reinstall. 2024-05-08T21:32:11.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum reinstall -y kernel || true 2024-05-08T21:32:11.474 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:32:11.474 INFO:teuthology.orchestra.run.smithi064.stdout:Unable to read consumer identity 2024-05-08T21:32:11.713 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:32:11.713 INFO:teuthology.orchestra.run.smithi064.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T21:32:11.713 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:32:12.040 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:04 ago on Wed 08 May 2024 09:32:08 PM UTC. 2024-05-08T21:32:12.116 INFO:teuthology.orchestra.run.smithi064.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T21:32:12.119 INFO:teuthology.orchestra.run.smithi064.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T21:32:12.157 INFO:teuthology.orchestra.run.smithi064.stderr:Error: No packages marked for reinstall. 2024-05-08T21:32:12.217 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T21:32:12.377 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T21:32:12.377 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T21:32:12.378 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T21:32:12.378 DEBUG:teuthology.parallel:result is None 2024-05-08T21:32:12.378 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T21:32:12.541 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T21:32:12.542 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T21:32:12.546 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T21:32:12.551 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T21:32:12.574 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T21:32:12.780 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T21:32:12.800 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T21:32:12.801 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T21:32:12.803 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T21:32:12.806 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T21:32:12.857 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T21:32:12.863 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T21:32:12.863 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T21:32:12.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:32:12.896 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T21:32:12.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:32:12.915 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T21:32:12.917 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T21:32:12.938 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T21:32:12.945 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:32:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:32:12.981 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:32:12.996 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:32:12.999 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:32:13.020 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:32:13.022 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T21:32:13.030 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T21:32:13.030 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T21:32:13.033 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T21:32:13.035 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T21:32:13.093 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T21:32:13.104 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T21:32:13.105 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T21:32:13.108 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T21:32:13.110 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T21:32:13.151 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:32:13.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:32:13.366 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:32:13.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:32:13.453 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:32:13.453 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T21:32:13.521 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:32:13.566 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:32:13.632 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:32:13.675 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:32:13.723 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:32:13.723 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T21:32:13.793 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:32:13.837 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:32:13.895 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:32:13.976 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:32:14.027 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:32:14.027 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T21:32:14.096 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-05-08T21:32:14.099 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-05-08T21:32:14.101 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-05-08T21:32:14.143 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:32:14.154 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:32:14.194 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:32:14.655 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T21:32:14.663 INFO:teuthology.task.internal:Starting timer... 2024-05-08T21:32:14.663 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T21:32:14.674 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T21:32:14.682 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T21:32:14.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-05-08T21:32:14.747 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-05-08T21:32:14.749 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-05-08T21:32:14.828 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-05-08T21:32:14.830 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-05-08T21:32:14.885 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-05-08T21:32:14.886 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T21:32:14.886 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-05-08T21:32:14.908 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-05-08T21:32:14.909 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-05-08T21:32:14.927 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-05-08T21:32:14.928 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-05-08T21:32:14.955 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-05-08T21:32:14.955 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-05-08T21:32:14.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:32:14.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:32:14.986 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:32:15.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:32:15.013 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:32:15.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:32:15.040 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T21:32:15.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-05-08T21:32:15.069 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-05-08T21:32:15.114 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-05-08T21:32:15.159 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T21:32:15.168 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T21:32:15.188 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T21:32:15.190 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi160.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T21:42:48.315 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-05-08T21:42:48.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-08T21:42:48.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:42:48.397 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-05-08T21:42:48.465 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-08T21:42:48.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-08T21:42:48.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:42:48.538 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-08T21:42:48.607 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-08T21:42:48.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-05-08T21:42:48.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:42:48.683 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-05-08T21:42:48.751 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-05-08T21:42:48.751 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T21:42:48.764 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T21:42:48.764 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:42:48.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:42:48.767 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:42:48.767 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:42:48.769 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:42:48.769 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:42:48.800 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:42:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:42:48.816 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:42:48.817 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:42:48.824 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:42:48.839 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:42:48.884 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-05-08T21:42:48.895 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-05-08T21:42:48.896 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-05-08T21:42:48.909 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-05-08T21:42:48.911 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:42:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:42:48.925 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:42:48.929 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-05-08T21:42:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:42:48.940 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-05-08T21:42:48.955 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:42:48.969 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:42:48.995 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2024-05-08T21:42:48.999 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:42:48.999 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-08T21:42:48.999 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:48.999 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:48.999 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:48.999 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-05-08T21:42:49.031 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:42:49.032 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-08T21:42:49.032 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.032 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.032 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.032 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.048 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2024-05-08T21:42:49.052 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:42:49.052 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-05-08T21:42:49.052 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.052 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.052 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.052 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:42:49.053 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T21:42:49.063 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T21:42:49.064 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:42:49.064 DEBUG:teuthology.orchestra.run.smithi064:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:42:49.064 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi160.front.sepia.ceph.com 2024-05-08T21:42:49.064 DEBUG:teuthology.orchestra.run.smithi160:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:42:49.065 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi186.front.sepia.ceph.com 2024-05-08T21:42:49.065 DEBUG:teuthology.orchestra.run.smithi186:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:42:49.441 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T21:42:49.470 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:42:49.472 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2024-05-08T21:42:51.248 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-05-08T21:42:51.381 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 kB/s | 4.1 kB 00:00 2024-05-08T21:42:51.480 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-05-08T21:42:51.558 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 51 kB/s | 4.5 kB 00:00 2024-05-08T21:42:51.763 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-05-08T21:42:51.892 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-05-08T21:42:54.731 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-05-08T21:42:54.732 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-05-08T21:42:54.732 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2024-05-08T21:42:54.732 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-05-08T21:42:54.732 INFO:teuthology.orchestra.run.smithi186.stdout:Resetting modules: 2024-05-08T21:42:54.733 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools 2024-05-08T21:42:54.733 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:42:54.733 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-05-08T21:42:54.733 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-05-08T21:42:54.733 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:42:54.892 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-08T21:42:54.893 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout:Resetting modules: 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-08T21:42:54.894 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:42:55.335 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-08T21:42:55.337 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-05-08T21:42:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-05-08T21:42:55.337 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-05-08T21:42:55.337 INFO:teuthology.orchestra.run.smithi064.stdout:Resetting modules: 2024-05-08T21:42:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools 2024-05-08T21:42:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:42:55.338 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-08T21:42:55.338 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-05-08T21:42:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:43:08.139 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-08T21:43:08.649 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:08.813 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-05-08T21:43:08.916 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-08T21:43:09.284 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:09.414 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:10.394 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 kB/s | 4.1 kB 00:00 2024-05-08T21:43:10.813 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-05-08T21:43:11.258 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-05-08T21:43:11.357 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-05-08T21:43:11.638 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-05-08T21:43:11.743 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 kB/s | 4.5 kB 00:00 2024-05-08T21:43:13.854 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-08T21:43:14.024 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2024-05-08T21:43:14.024 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout:Installing group/module packages: 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:43:14.025 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout:Installing module profiles: 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools/common 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools rhel8 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout:Install 13 Packages 2024-05-08T21:43:14.026 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:43:14.027 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 26 M 2024-05-08T21:43:14.027 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 87 M 2024-05-08T21:43:14.027 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-08T21:43:14.182 INFO:teuthology.orchestra.run.smithi160.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 255 kB/s | 39 kB 00:00 2024-05-08T21:43:14.207 INFO:teuthology.orchestra.run.smithi160.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 138 kB/s | 25 kB 00:00 2024-05-08T21:43:14.232 INFO:teuthology.orchestra.run.smithi160.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 458 kB/s | 94 kB 00:00 2024-05-08T21:43:14.377 INFO:teuthology.orchestra.run.smithi160.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 5.9 MB/s | 1.0 MB 00:00 2024-05-08T21:43:14.400 INFO:teuthology.orchestra.run.smithi160.stdout:(5/13): yajl-2.1.0-12.el8.x86_64.rpm 188 kB/s | 41 kB 00:00 2024-05-08T21:43:14.559 INFO:teuthology.orchestra.run.smithi160.stdout:(6/13): buildah-1.31.5-1.module+el8.9.0+21697+6 27 MB/s | 8.8 MB 00:00 2024-05-08T21:43:14.584 INFO:teuthology.orchestra.run.smithi160.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.3 MB/s | 239 kB 00:00 2024-05-08T21:43:14.617 INFO:teuthology.orchestra.run.smithi160.stdout:(8/13): cockpit-podman-75-1.module+el8.9.0+2169 3.0 MB/s | 739 kB 00:00 2024-05-08T21:43:14.742 INFO:teuthology.orchestra.run.smithi160.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 20 MB/s | 3.7 MB 00:00 2024-05-08T21:43:14.768 INFO:teuthology.orchestra.run.smithi160.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 834 kB/s | 154 kB 00:00 2024-05-08T21:43:14.806 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-05-08T21:43:14.868 INFO:teuthology.orchestra.run.smithi160.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 20 MB/s | 2.5 MB 00:00 2024-05-08T21:43:14.893 INFO:teuthology.orchestra.run.smithi160.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 395 kB/s | 49 kB 00:00 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout:========================================================================================================== 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout:========================================================================================================== 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout:Installing group/module packages: 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:43:14.974 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout:Installing module profiles: 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools/common 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools rhel8 2024-05-08T21:43:14.975 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout:========================================================================================================== 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout:Install 13 Packages 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 26 M 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 87 M 2024-05-08T21:43:14.976 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-05-08T21:43:15.019 INFO:teuthology.orchestra.run.smithi160.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T21:43:15.019 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:43:15.019 INFO:teuthology.orchestra.run.smithi160.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-08T21:43:15.120 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-08T21:43:15.136 INFO:teuthology.orchestra.run.smithi186.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 156 kB/s | 25 kB 00:00 2024-05-08T21:43:15.181 INFO:teuthology.orchestra.run.smithi186.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 193 kB/s | 39 kB 00:00 2024-05-08T21:43:15.211 INFO:teuthology.orchestra.run.smithi186.stdout:(3/13): yajl-2.1.0-12.el8.x86_64.rpm 544 kB/s | 41 kB 00:00 2024-05-08T21:43:15.237 INFO:teuthology.orchestra.run.smithi186.stdout:(4/13): python3-pyxdg-0.25-16.el8.noarch.rpm 361 kB/s | 94 kB 00:00 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout:========================================================================================================== 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout:========================================================================================================== 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout:Installing group/module packages: 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:43:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout:Installing module profiles: 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools/common 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:43:15.297 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2024-05-08T21:43:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools rhel8 2024-05-08T21:43:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:43:15.298 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-08T21:43:15.298 INFO:teuthology.orchestra.run.smithi064.stdout:========================================================================================================== 2024-05-08T21:43:15.298 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-05-08T21:43:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:43:15.299 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 26 M 2024-05-08T21:43:15.299 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 87 M 2024-05-08T21:43:15.299 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-05-08T21:43:15.299 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-08T21:43:15.320 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-08T21:43:15.320 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-08T21:43:15.320 INFO:teuthology.orchestra.run.smithi186.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 7.2 MB/s | 1.0 MB 00:00 2024-05-08T21:43:15.371 INFO:teuthology.orchestra.run.smithi186.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 5.4 MB/s | 739 kB 00:00 2024-05-08T21:43:15.428 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 192 kB/s | 25 kB 00:00 2024-05-08T21:43:15.429 INFO:teuthology.orchestra.run.smithi186.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.2 MB/s | 239 kB 00:00 2024-05-08T21:43:15.454 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 608 kB/s | 94 kB 00:00 2024-05-08T21:43:15.479 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 219 kB/s | 39 kB 00:00 2024-05-08T21:43:15.521 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 443 kB/s | 41 kB 00:00 2024-05-08T21:43:15.550 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-08T21:43:15.558 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-08T21:43:15.571 INFO:teuthology.orchestra.run.smithi186.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 25 MB/s | 8.8 MB 00:00 2024-05-08T21:43:15.621 INFO:teuthology.orchestra.run.smithi186.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 801 kB/s | 154 kB 00:00 2024-05-08T21:43:15.663 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 4.8 MB/s | 1.0 MB 00:00 2024-05-08T21:43:15.688 INFO:teuthology.orchestra.run.smithi186.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 12 MB/s | 3.7 MB 00:00 2024-05-08T21:43:15.763 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): buildah-1.31.5-1.module+el8.9.0+21697+6 31 MB/s | 8.8 MB 00:00 2024-05-08T21:43:15.780 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.0 MB/s | 239 kB 00:00 2024-05-08T21:43:15.808 INFO:teuthology.orchestra.run.smithi186.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 14 MB/s | 2.5 MB 00:00 2024-05-08T21:43:15.814 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): cockpit-podman-75-1.module+el8.9.0+2169 2.5 MB/s | 739 kB 00:00 2024-05-08T21:43:15.830 INFO:teuthology.orchestra.run.smithi186.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 348 kB/s | 49 kB 00:00 2024-05-08T21:43:15.922 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.1 MB/s | 154 kB 00:00 2024-05-08T21:43:15.939 INFO:teuthology.orchestra.run.smithi186.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 22 MB/s | 8.1 MB 00:00 2024-05-08T21:43:15.940 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:43:15.940 INFO:teuthology.orchestra.run.smithi186.stdout:Total 27 MB/s | 26 MB 00:00 2024-05-08T21:43:16.039 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 13 MB/s | 3.7 MB 00:00 2024-05-08T21:43:16.165 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 23 MB/s | 8.1 MB 00:00 2024-05-08T21:43:16.213 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-05-08T21:43:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 8.2 MB/s | 2.5 MB 00:00 2024-05-08T21:43:16.235 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-05-08T21:43:16.235 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-05-08T21:43:16.248 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+21697+6a 237 kB/s | 49 kB 00:00 2024-05-08T21:43:16.249 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:43:16.249 INFO:teuthology.orchestra.run.smithi064.stdout:Total 27 MB/s | 26 MB 00:00 2024-05-08T21:43:16.312 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-08T21:43:16.464 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-05-08T21:43:16.472 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-05-08T21:43:16.542 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-05-08T21:43:16.565 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-05-08T21:43:16.565 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-05-08T21:43:16.810 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-05-08T21:43:16.818 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-05-08T21:43:16.839 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:43:17.133 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:43:17.186 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-05-08T21:43:17.321 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:43:17.489 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:43:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-05-08T21:43:17.676 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:43:17.682 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:43:17.889 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:43:17.926 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:43:18.056 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:43:18.145 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:43:18.145 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:43:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:43:18.310 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:43:18.419 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:43:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:43:18.628 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:43:18.631 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:43:18.792 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:43:18.872 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:43:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:43:19.008 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:43:19.238 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:43:19.263 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:43:19.638 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:43:19.747 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:43:19.969 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:43:20.148 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:43:20.374 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:43:20.381 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:43:20.546 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:43:20.791 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:43:20.801 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:43:20.991 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:43:21.338 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:43:21.405 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:43:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:43:21.535 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:43:21.726 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:43:21.731 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:43:21.951 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:43:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:43:22.137 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:43:22.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:43:22.539 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:43:22.540 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:43:35.710 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:43:35.710 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:43:36.276 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.277 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.277 INFO:teuthology.orchestra.run.smithi186.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:36.277 INFO:teuthology.orchestra.run.smithi186.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:43:36.277 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:43:36.277 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-05-08T21:43:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:43:36.340 INFO:teuthology.orchestra.run.smithi064.stdout:Installed products updated. 2024-05-08T21:43:36.441 DEBUG:teuthology.parallel:result is None 2024-05-08T21:43:36.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:43:36.942 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-05-08T21:43:36.942 INFO:teuthology.orchestra.run.smithi064.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:43:36.943 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-08T21:43:37.125 DEBUG:teuthology.parallel:result is None 2024-05-08T21:43:38.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:43:38.649 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:39.229 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:43:39.230 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-08T21:43:39.402 DEBUG:teuthology.parallel:result is None 2024-05-08T21:43:39.402 INFO:teuthology.run_tasks:Running task install... 2024-05-08T21:43:39.411 DEBUG:teuthology.task.install:project ceph 2024-05-08T21:43:39.411 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T21:43:39.411 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T21:43:39.412 INFO:teuthology.task.install:Using flavor: default 2024-05-08T21:43:39.418 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T21:43:39.418 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T21:43:39.418 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:43:39.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:43:39.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:43:39.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:43:39.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:43:39.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:43:39.579 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:43:39.579 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:43:39.646 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:43:39.647 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:43:39.651 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:43:39.651 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:43:39.863 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:43:39.863 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:43:39.864 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:43:39.891 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:43:39.892 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:43:39.944 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:43:39.944 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:43:39.944 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:43:39.948 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:43:39.948 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:43:39.948 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:43:39.960 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:43:39.978 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:43:39.979 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:43:39.981 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:43:39.981 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:43:40.037 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:43:40.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:43:40.053 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:43:40.104 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-05-08T21:43:40.105 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-05-08T21:43:40.134 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:43:40.135 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:43:40.202 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-05-08T21:43:40.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-05-08T21:43:40.205 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-05-08T21:43:40.205 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-05-08T21:43:40.484 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:40.584 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:40.603 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:41.880 INFO:teuthology.orchestra.run.smithi064.stdout:63 files removed 2024-05-08T21:43:41.898 INFO:teuthology.orchestra.run.smithi160.stdout:63 files removed 2024-05-08T21:43:41.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:43:41.950 INFO:teuthology.orchestra.run.smithi186.stdout:63 files removed 2024-05-08T21:43:41.955 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:43:42.001 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:43:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:42.342 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2024-05-08T21:43:44.081 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 174 kB/s | 80 kB 00:00 2024-05-08T21:43:44.205 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 199 kB/s | 80 kB 00:00 2024-05-08T21:43:44.416 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-05-08T21:43:44.486 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 203 kB/s | 80 kB 00:00 2024-05-08T21:43:44.605 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-05-08T21:43:44.716 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 5.4 kB/s | 1.5 kB 00:00 2024-05-08T21:43:44.829 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-05-08T21:43:44.900 INFO:teuthology.orchestra.run.smithi160.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T21:43:44.973 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2024-05-08T21:43:45.096 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 6.2 kB/s | 1.5 kB 00:00 2024-05-08T21:43:45.115 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2024-05-08T21:43:45.249 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-05-08T21:43:46.027 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-05-08T21:43:47.051 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2024-05-08T21:43:47.202 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 16 MB 00:02 2024-05-08T21:43:49.683 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 398 kB/s | 24 kB 00:00 2024-05-08T21:43:50.684 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 411 kB/s | 24 kB 00:00 2024-05-08T21:43:51.010 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 67 MB 00:01 2024-05-08T21:43:51.041 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 479 kB/s | 24 kB 00:00 2024-05-08T21:43:52.153 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 67 MB 00:01 2024-05-08T21:43:52.901 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 MB/s | 67 MB 00:01 2024-05-08T21:44:02.211 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 61 MB 00:01 2024-05-08T21:44:02.362 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-08T21:44:03.860 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-05-08T21:44:05.413 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 61 MB 00:01 2024-05-08T21:44:07.028 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 61 MB 00:01 2024-05-08T21:44:13.253 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-08T21:44:20.386 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:44:20.387 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:44:20.405 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:44:20.407 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:44:20.408 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:44:20.409 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:44:20.412 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:44:20.416 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:44:20.418 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:44:20.640 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout:=================================================================================================================================== 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout:=================================================================================================================================== 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:44:20.654 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:44:20.655 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:44:20.656 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:44:20.657 INFO:teuthology.orchestra.run.smithi186.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:44:20.658 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:44:20.659 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:44:20.660 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:44:20.661 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:44:20.662 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:44:20.663 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:44:20.664 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2024-05-08T21:44:20.665 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout:=================================================================================================================================== 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout:Install 130 Packages 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 16 Packages 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 215 M 2024-05-08T21:44:20.666 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-05-08T21:44:20.986 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:44:20.987 INFO:teuthology.orchestra.run.smithi186.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2024-05-08T21:44:20.988 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:44:21.007 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:44:21.009 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:44:21.010 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:44:21.012 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:44:21.015 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:44:21.019 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:44:21.021 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:44:21.244 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout:=================================================================================================================================== 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout:=================================================================================================================================== 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:44:21.258 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:44:21.259 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:44:21.260 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:44:21.261 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:44:21.262 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:44:21.263 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:44:21.264 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:44:21.265 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:44:21.266 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:44:21.267 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:44:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql 10 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout:=================================================================================================================================== 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout:Install 130 Packages 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 16 Packages 2024-05-08T21:44:21.269 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:44:21.270 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 215 M 2024-05-08T21:44:21.270 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-08T21:44:21.371 INFO:teuthology.orchestra.run.smithi186.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 2.2 MB/s | 860 kB 00:00 2024-05-08T21:44:21.454 INFO:teuthology.orchestra.run.smithi186.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.9 MB/s | 166 kB 00:00 2024-05-08T21:44:21.588 INFO:teuthology.orchestra.run.smithi160.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2024-05-08T21:44:21.763 INFO:teuthology.orchestra.run.smithi186.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 5.7 MB/s | 5.5 MB 00:00 2024-05-08T21:44:22.038 INFO:teuthology.orchestra.run.smithi186.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 5.7 MB/s | 1.6 MB 00:00 2024-05-08T21:44:22.180 INFO:teuthology.orchestra.run.smithi186.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 3.1 MB/s | 2.3 MB 00:00 2024-05-08T21:44:22.531 INFO:teuthology.orchestra.run.smithi186.stdout:(7/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 13 MB/s | 22 MB 00:01 2024-05-08T21:44:22.541 INFO:teuthology.orchestra.run.smithi160.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 902 kB/s | 860 kB 00:00 2024-05-08T21:44:22.765 INFO:teuthology.orchestra.run.smithi186.stdout:(8/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 6.7 MB/s | 4.8 MB 00:00 2024-05-08T21:44:22.780 INFO:teuthology.orchestra.run.smithi160.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 697 kB/s | 166 kB 00:00 2024-05-08T21:44:22.831 INFO:teuthology.orchestra.run.smithi186.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 366 kB/s | 24 kB 00:00 2024-05-08T21:44:23.740 INFO:teuthology.orchestra.run.smithi186.stdout:(10/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 11 MB/s | 13 MB 00:01 2024-05-08T21:44:23.823 INFO:teuthology.orchestra.run.smithi186.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 3.1 MB/s | 261 kB 00:00 2024-05-08T21:44:23.890 INFO:teuthology.orchestra.run.smithi186.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 465 kB/s | 31 kB 00:00 2024-05-08T21:44:23.911 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:44:23.912 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:44:23.931 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:44:23.933 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:44:23.934 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:44:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:44:23.939 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:44:23.943 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:44:23.944 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:44:23.990 INFO:teuthology.orchestra.run.smithi186.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 7.6 MB/s | 782 kB 00:00 2024-05-08T21:44:24.066 INFO:teuthology.orchestra.run.smithi186.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.8 MB/s | 142 kB 00:00 2024-05-08T21:44:24.141 INFO:teuthology.orchestra.run.smithi186.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.8 MB/s | 139 kB 00:00 2024-05-08T21:44:24.177 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-08T21:44:24.192 INFO:teuthology.orchestra.run.smithi064.stdout:=================================================================================================================================== 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout:=================================================================================================================================== 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:44:24.193 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:44:24.194 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:44:24.195 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:44:24.196 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:44:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:44:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:44:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:44:24.200 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:44:24.201 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:44:24.202 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:44:24.203 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:44:24.204 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql 10 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout:=================================================================================================================================== 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout:Install 130 Packages 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 16 Packages 2024-05-08T21:44:24.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:44:24.206 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 215 M 2024-05-08T21:44:24.206 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-05-08T21:44:24.241 INFO:teuthology.orchestra.run.smithi186.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 4.6 MB/s | 474 kB 00:00 2024-05-08T21:44:24.458 INFO:teuthology.orchestra.run.smithi186.stdout:(17/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 8.1 MB/s | 18 MB 00:02 2024-05-08T21:44:24.567 INFO:teuthology.orchestra.run.smithi186.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 413 kB/s | 45 kB 00:00 2024-05-08T21:44:24.570 INFO:teuthology.orchestra.run.smithi064.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2024-05-08T21:44:24.605 INFO:teuthology.orchestra.run.smithi160.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 1.7 MB/s | 5.5 MB 00:03 2024-05-08T21:44:24.750 INFO:teuthology.orchestra.run.smithi186.stdout:(19/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 13 MB/s | 6.5 MB 00:00 2024-05-08T21:44:24.776 INFO:teuthology.orchestra.run.smithi186.stdout:(20/146): python3-ceph-common-18.2.2-1322.gd21e 618 kB/s | 129 kB 00:00 2024-05-08T21:44:24.801 INFO:teuthology.orchestra.run.smithi186.stdout:(21/146): python3-cephfs-18.2.2-1322.gd21e4f96. 3.8 MB/s | 197 kB 00:00 2024-05-08T21:44:24.804 INFO:teuthology.orchestra.run.smithi064.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 3.6 MB/s | 860 kB 00:00 2024-05-08T21:44:24.860 INFO:teuthology.orchestra.run.smithi186.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 6.1 MB/s | 365 kB 00:00 2024-05-08T21:44:24.871 INFO:teuthology.orchestra.run.smithi064.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.4 MB/s | 166 kB 00:00 2024-05-08T21:44:24.885 INFO:teuthology.orchestra.run.smithi186.stdout:(23/146): python3-rados-18.2.2-1322.gd21e4f96.e 3.3 MB/s | 370 kB 00:00 2024-05-08T21:44:24.902 INFO:teuthology.orchestra.run.smithi186.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 2.5 MB/s | 107 kB 00:00 2024-05-08T21:44:24.994 INFO:teuthology.orchestra.run.smithi186.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 876 kB/s | 95 kB 00:00 2024-05-08T21:44:25.152 INFO:teuthology.orchestra.run.smithi186.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.1 MB/s | 181 kB 00:00 2024-05-08T21:44:25.164 INFO:teuthology.orchestra.run.smithi160.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 2.8 MB/s | 1.6 MB 00:00 2024-05-08T21:44:25.238 INFO:teuthology.orchestra.run.smithi064.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 6.2 MB/s | 2.3 MB 00:00 2024-05-08T21:44:25.278 INFO:teuthology.orchestra.run.smithi186.stdout:(27/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 9.2 MB/s | 3.5 MB 00:00 2024-05-08T21:44:25.303 INFO:teuthology.orchestra.run.smithi186.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 161 kB/s | 24 kB 00:00 2024-05-08T21:44:25.345 INFO:teuthology.orchestra.run.smithi186.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.0 MB/s | 138 kB 00:00 2024-05-08T21:44:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 8.1 MB/s | 1.6 MB 00:00 2024-05-08T21:44:25.490 INFO:teuthology.orchestra.run.smithi160.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 858 kB/s | 2.3 MB 00:02 2024-05-08T21:44:25.737 INFO:teuthology.orchestra.run.smithi186.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 8.1 MB/s | 3.5 MB 00:00 2024-05-08T21:44:25.837 INFO:teuthology.orchestra.run.smithi186.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.4 MB/s | 245 kB 00:00 2024-05-08T21:44:25.929 INFO:teuthology.orchestra.run.smithi186.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 536 kB/s | 49 kB 00:00 2024-05-08T21:44:26.021 INFO:teuthology.orchestra.run.smithi186.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 178 kB/s | 16 kB 00:00 2024-05-08T21:44:26.113 INFO:teuthology.orchestra.run.smithi186.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.8 MB/s | 263 kB 00:00 2024-05-08T21:44:26.205 INFO:teuthology.orchestra.run.smithi186.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.8 MB/s | 170 kB 00:00 2024-05-08T21:44:26.355 INFO:teuthology.orchestra.run.smithi186.stdout:(36/146): ceph-mgr-diskprediction-local-18.2.2- 7.3 MB/s | 7.4 MB 00:01 2024-05-08T21:44:26.372 INFO:teuthology.orchestra.run.smithi186.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 108 kB/s | 18 kB 00:00 2024-05-08T21:44:26.547 INFO:teuthology.orchestra.run.smithi186.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 336 kB/s | 59 kB 00:00 2024-05-08T21:44:26.564 INFO:teuthology.orchestra.run.smithi186.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-05-08T21:44:26.631 INFO:teuthology.orchestra.run.smithi186.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 898 kB/s | 75 kB 00:00 2024-05-08T21:44:26.657 INFO:teuthology.orchestra.run.smithi186.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 5.0 MB/s | 476 kB 00:00 2024-05-08T21:44:26.690 INFO:teuthology.orchestra.run.smithi186.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 744 kB/s | 44 kB 00:00 2024-05-08T21:44:26.715 INFO:teuthology.orchestra.run.smithi186.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2024-05-08T21:44:26.766 INFO:teuthology.orchestra.run.smithi186.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2024-05-08T21:44:26.791 INFO:teuthology.orchestra.run.smithi186.stdout:(45/146): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2024-05-08T21:44:26.825 INFO:teuthology.orchestra.run.smithi186.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2024-05-08T21:44:26.883 INFO:teuthology.orchestra.run.smithi186.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2024-05-08T21:44:26.909 INFO:teuthology.orchestra.run.smithi186.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 973 kB/s | 81 kB 00:00 2024-05-08T21:44:26.942 INFO:teuthology.orchestra.run.smithi186.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2024-05-08T21:44:26.947 INFO:teuthology.orchestra.run.smithi160.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.7 MB/s | 4.8 MB 00:01 2024-05-08T21:44:26.968 INFO:teuthology.orchestra.run.smithi186.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 299 kB/s | 18 kB 00:00 2024-05-08T21:44:27.035 INFO:teuthology.orchestra.run.smithi186.stdout:(51/146): python3-kubernetes-11.0.0-6.el8.noarc 13 MB/s | 1.2 MB 00:00 2024-05-08T21:44:27.060 INFO:teuthology.orchestra.run.smithi186.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 530 kB/s | 49 kB 00:00 2024-05-08T21:44:27.094 INFO:teuthology.orchestra.run.smithi186.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2024-05-08T21:44:27.136 INFO:teuthology.orchestra.run.smithi186.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 801 kB/s | 60 kB 00:00 2024-05-08T21:44:27.161 INFO:teuthology.orchestra.run.smithi186.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2024-05-08T21:44:27.171 INFO:teuthology.orchestra.run.smithi064.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 2.0 MB/s | 5.5 MB 00:02 2024-05-08T21:44:27.203 INFO:teuthology.orchestra.run.smithi186.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2024-05-08T21:44:27.237 INFO:teuthology.orchestra.run.smithi186.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2024-05-08T21:44:27.262 INFO:teuthology.orchestra.run.smithi186.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2024-05-08T21:44:27.304 INFO:teuthology.orchestra.run.smithi186.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 961 kB/s | 64 kB 00:00 2024-05-08T21:44:27.337 INFO:teuthology.orchestra.run.smithi186.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2024-05-08T21:44:27.362 INFO:teuthology.orchestra.run.smithi186.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2024-05-08T21:44:27.400 INFO:teuthology.orchestra.run.smithi186.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2024-05-08T21:44:27.430 INFO:teuthology.orchestra.run.smithi186.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2024-05-08T21:44:27.455 INFO:teuthology.orchestra.run.smithi186.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2024-05-08T21:44:27.530 INFO:teuthology.orchestra.run.smithi186.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 815 kB/s | 61 kB 00:00 2024-05-08T21:44:27.556 INFO:teuthology.orchestra.run.smithi186.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2024-05-08T21:44:27.581 INFO:teuthology.orchestra.run.smithi186.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2024-05-08T21:44:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.2 MB/s | 4.8 MB 00:02 2024-05-08T21:44:27.648 INFO:teuthology.orchestra.run.smithi186.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 250 kB/s | 23 kB 00:00 2024-05-08T21:44:27.690 INFO:teuthology.orchestra.run.smithi186.stdout:(69/146): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-05-08T21:44:27.715 INFO:teuthology.orchestra.run.smithi186.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00 2024-05-08T21:44:27.832 INFO:teuthology.orchestra.run.smithi186.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 126 kB/s | 18 kB 00:00 2024-05-08T21:44:27.872 INFO:teuthology.orchestra.run.smithi064.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 6.2 MB/s | 22 MB 00:03 2024-05-08T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 251 kB/s | 43 kB 00:00 2024-05-08T21:44:27.930 INFO:teuthology.orchestra.run.smithi064.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 418 kB/s | 24 kB 00:00 2024-05-08T21:44:27.966 INFO:teuthology.orchestra.run.smithi186.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.1 MB/s | 155 kB 00:00 2024-05-08T21:44:28.083 INFO:teuthology.orchestra.run.smithi186.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 568 kB/s | 109 kB 00:00 2024-05-08T21:44:28.108 INFO:teuthology.orchestra.run.smithi186.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 301 kB/s | 43 kB 00:00 2024-05-08T21:44:28.217 INFO:teuthology.orchestra.run.smithi186.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 140 kB/s | 19 kB 00:00 2024-05-08T21:44:28.234 INFO:teuthology.orchestra.run.smithi186.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 350 kB/s | 44 kB 00:00 2024-05-08T21:44:28.359 INFO:teuthology.orchestra.run.smithi186.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.2 MB/s | 605 kB 00:00 2024-05-08T21:44:28.419 INFO:teuthology.orchestra.run.smithi186.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2024-05-08T21:44:28.518 INFO:teuthology.orchestra.run.smithi186.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-05-08T21:44:28.543 INFO:teuthology.orchestra.run.smithi186.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.6 MB/s | 200 kB 00:00 2024-05-08T21:44:28.661 INFO:teuthology.orchestra.run.smithi186.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 339 kB/s | 48 kB 00:00 2024-05-08T21:44:28.686 INFO:teuthology.orchestra.run.smithi186.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 612 kB/s | 87 kB 00:00 2024-05-08T21:44:28.786 INFO:teuthology.orchestra.run.smithi186.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-05-08T21:44:28.820 INFO:teuthology.orchestra.run.smithi186.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 289 kB/s | 39 kB 00:00 2024-05-08T21:44:28.887 INFO:teuthology.orchestra.run.smithi186.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 829 kB/s | 83 kB 00:00 2024-05-08T21:44:28.920 INFO:teuthology.orchestra.run.smithi186.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 6.3 MB/s | 645 kB 00:00 2024-05-08T21:44:29.004 INFO:teuthology.orchestra.run.smithi186.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2024-05-08T21:44:29.079 INFO:teuthology.orchestra.run.smithi186.stdout:(89/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-08T21:44:29.154 INFO:teuthology.orchestra.run.smithi186.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 484 kB/s | 73 kB 00:00 2024-05-08T21:44:29.246 INFO:teuthology.orchestra.run.smithi186.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 370 kB/s | 62 kB 00:00 2024-05-08T21:44:29.280 INFO:teuthology.orchestra.run.smithi186.stdout:(92/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.4 MB/s | 562 kB 00:00 2024-05-08T21:44:29.405 INFO:teuthology.orchestra.run.smithi186.stdout:(93/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-05-08T21:44:29.431 INFO:teuthology.orchestra.run.smithi186.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 313 kB/s | 47 kB 00:00 2024-05-08T21:44:29.556 INFO:teuthology.orchestra.run.smithi186.stdout:(95/146): NetworkManager-initscripts-updown-1.4 1.1 MB/s | 144 kB 00:00 2024-05-08T21:44:29.590 INFO:teuthology.orchestra.run.smithi186.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.0 MB/s | 559 kB 00:00 2024-05-08T21:44:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 7.5 MB/s | 18 MB 00:02 2024-05-08T21:44:29.673 INFO:teuthology.orchestra.run.smithi186.stdout:(97/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 855 kB/s | 100 kB 00:00 2024-05-08T21:44:29.722 INFO:teuthology.orchestra.run.smithi064.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 3.1 MB/s | 261 kB 00:00 2024-05-08T21:44:29.732 INFO:teuthology.orchestra.run.smithi186.stdout:(98/146): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2024-05-08T21:44:29.789 INFO:teuthology.orchestra.run.smithi064.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 464 kB/s | 31 kB 00:00 2024-05-08T21:44:29.815 INFO:teuthology.orchestra.run.smithi186.stdout:(99/146): python3-prettytable-0.7.2-14.el8.noar 310 kB/s | 44 kB 00:00 2024-05-08T21:44:29.866 INFO:teuthology.orchestra.run.smithi186.stdout:(100/146): python3-pyasn1-modules-0.3.7-6.el8.n 824 kB/s | 110 kB 00:00 2024-05-08T21:44:29.908 INFO:teuthology.orchestra.run.smithi186.stdout:(101/146): python3-jsonpatch-1.21-2.el8.noarch. 289 kB/s | 27 kB 00:00 2024-05-08T21:44:29.956 INFO:teuthology.orchestra.run.smithi064.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 4.6 MB/s | 782 kB 00:00 2024-05-08T21:44:29.974 INFO:teuthology.orchestra.run.smithi186.stdout:(102/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.1 MB/s | 126 kB 00:00 2024-05-08T21:44:30.016 INFO:teuthology.orchestra.run.smithi186.stdout:(103/146): python3-werkzeug-0.12.2-4.el8.noarch 4.1 MB/s | 457 kB 00:00 2024-05-08T21:44:30.031 INFO:teuthology.orchestra.run.smithi064.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.8 MB/s | 142 kB 00:00 2024-05-08T21:44:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.6 MB/s | 139 kB 00:00 2024-05-08T21:44:30.125 INFO:teuthology.orchestra.run.smithi186.stdout:(104/146): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2024-05-08T21:44:30.168 INFO:teuthology.orchestra.run.smithi186.stdout:(105/146): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2024-05-08T21:44:30.207 INFO:teuthology.orchestra.run.smithi064.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 5.1 MB/s | 474 kB 00:00 2024-05-08T21:44:30.317 INFO:teuthology.orchestra.run.smithi186.stdout:(106/146): python3-markupsafe-0.23-19.el8.x86_6 259 kB/s | 39 kB 00:00 2024-05-08T21:44:30.380 INFO:teuthology.orchestra.run.smithi186.stdout:(107/146): libtool-2.4.6-25.el8.x86_64.rpm 2.7 MB/s | 709 kB 00:00 2024-05-08T21:44:30.451 INFO:teuthology.orchestra.run.smithi186.stdout:(108/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2024-05-08T21:44:30.477 INFO:teuthology.orchestra.run.smithi186.stdout:(109/146): uuid-1.6.2-43.el8.x86_64.rpm 660 kB/s | 64 kB 00:00 2024-05-08T21:44:30.552 INFO:teuthology.orchestra.run.smithi186.stdout:(110/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.0 MB/s | 103 kB 00:00 2024-05-08T21:44:30.719 INFO:teuthology.orchestra.run.smithi186.stdout:(111/146): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-05-08T21:44:30.769 INFO:teuthology.orchestra.run.smithi186.stdout:(112/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.4 MB/s | 538 kB 00:00 2024-05-08T21:44:30.869 INFO:teuthology.orchestra.run.smithi186.stdout:(113/146): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-05-08T21:44:30.928 INFO:teuthology.orchestra.run.smithi186.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 18 MB/s | 3.7 MB 00:00 2024-05-08T21:44:30.982 INFO:teuthology.orchestra.run.smithi064.stdout:(17/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 8.4 MB/s | 6.5 MB 00:00 2024-05-08T21:44:31.029 INFO:teuthology.orchestra.run.smithi186.stdout:(115/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.4 MB/s | 225 kB 00:00 2024-05-08T21:44:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 487 kB/s | 45 kB 00:00 2024-05-08T21:44:31.104 INFO:teuthology.orchestra.run.smithi186.stdout:(116/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2024-05-08T21:44:31.150 INFO:teuthology.orchestra.run.smithi064.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.7 MB/s | 129 kB 00:00 2024-05-08T21:44:31.241 INFO:teuthology.orchestra.run.smithi064.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.1 MB/s | 197 kB 00:00 2024-05-08T21:44:31.279 INFO:teuthology.orchestra.run.smithi186.stdout:(117/146): openblas-threads-0.3.15-6.el8.x86_64 21 MB/s | 5.2 MB 00:00 2024-05-08T21:44:31.304 INFO:teuthology.orchestra.run.smithi186.stdout:(118/146): python3-mako-1.0.6-14.el8.noarch.rpm 785 kB/s | 157 kB 00:00 2024-05-08T21:44:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 3.6 MB/s | 370 kB 00:00 2024-05-08T21:44:31.425 INFO:teuthology.orchestra.run.smithi064.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 4.3 MB/s | 365 kB 00:00 2024-05-08T21:44:31.464 INFO:teuthology.orchestra.run.smithi160.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 2.9 MB/s | 13 MB 00:04 2024-05-08T21:44:31.514 INFO:teuthology.orchestra.run.smithi160.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 489 kB/s | 24 kB 00:00 2024-05-08T21:44:31.567 INFO:teuthology.orchestra.run.smithi064.stdout:(23/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 3.3 MB/s | 13 MB 00:03 2024-05-08T21:44:31.584 INFO:teuthology.orchestra.run.smithi064.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 672 kB/s | 107 kB 00:00 2024-05-08T21:44:31.618 INFO:teuthology.orchestra.run.smithi064.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.9 MB/s | 95 kB 00:00 2024-05-08T21:44:31.693 INFO:teuthology.orchestra.run.smithi064.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 2.4 MB/s | 181 kB 00:00 2024-05-08T21:44:31.768 INFO:teuthology.orchestra.run.smithi064.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 323 kB/s | 24 kB 00:00 2024-05-08T21:44:31.805 INFO:teuthology.orchestra.run.smithi186.stdout:(119/146): ceph-test-18.2.2-1322.gd21e4f96.el8. 5.3 MB/s | 47 MB 00:08 2024-05-08T21:44:31.846 INFO:teuthology.orchestra.run.smithi186.stdout:(120/146): automake-1.16.1-8.el8.noarch.rpm 1.3 MB/s | 714 kB 00:00 2024-05-08T21:44:31.852 INFO:teuthology.orchestra.run.smithi064.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.6 MB/s | 138 kB 00:00 2024-05-08T21:44:31.952 INFO:teuthology.orchestra.run.smithi064.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 9.4 MB/s | 3.5 MB 00:00 2024-05-08T21:44:31.972 INFO:teuthology.orchestra.run.smithi186.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 7.3 MB/s | 5.0 MB 00:00 2024-05-08T21:44:31.988 INFO:teuthology.orchestra.run.smithi186.stdout:(122/146): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-05-08T21:44:32.022 INFO:teuthology.orchestra.run.smithi186.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 912 kB/s | 198 kB 00:00 2024-05-08T21:44:32.097 INFO:teuthology.orchestra.run.smithi186.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 433 kB/s | 54 kB 00:00 2024-05-08T21:44:32.198 INFO:teuthology.orchestra.run.smithi186.stdout:(125/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-08T21:44:32.265 INFO:teuthology.orchestra.run.smithi186.stdout:(126/146): postgresql-10.23-4.module+el8.9.0+21 9.0 MB/s | 1.5 MB 00:00 2024-05-08T21:44:32.332 INFO:teuthology.orchestra.run.smithi186.stdout:(127/146): postgresql-contrib-10.23-4.module+el 5.9 MB/s | 812 kB 00:00 2024-05-08T21:44:32.465 INFO:teuthology.orchestra.run.smithi186.stdout:(128/146): postgresql-server-10.23-4.module+el8 25 MB/s | 5.1 MB 00:00 2024-05-08T21:44:32.490 INFO:teuthology.orchestra.run.smithi186.stdout:(129/146): python3-packaging-16.8-9.el8.noarch. 304 kB/s | 48 kB 00:00 2024-05-08T21:44:32.633 INFO:teuthology.orchestra.run.smithi186.stdout:(130/146): python3-scipy-1.0.0-21.module+el8.9. 21 MB/s | 14 MB 00:00 2024-05-08T21:44:32.711 INFO:teuthology.orchestra.run.smithi064.stdout:(30/146): ceph-mgr-diskprediction-local-18.2.2- 9.8 MB/s | 7.4 MB 00:00 2024-05-08T21:44:32.741 INFO:teuthology.orchestra.run.smithi186.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2024-05-08T21:44:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.9 MB/s | 245 kB 00:00 2024-05-08T21:44:32.895 INFO:teuthology.orchestra.run.smithi064.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 492 kB/s | 49 kB 00:00 2024-05-08T21:44:32.908 INFO:teuthology.orchestra.run.smithi186.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-05-08T21:44:32.970 INFO:teuthology.orchestra.run.smithi064.stdout:(33/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 3.2 MB/s | 3.5 MB 00:01 2024-05-08T21:44:32.987 INFO:teuthology.orchestra.run.smithi064.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1322.gd 177 kB/s | 16 kB 00:00 2024-05-08T21:44:33.025 INFO:teuthology.orchestra.run.smithi186.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 7.1 MB/s | 854 kB 00:00 2024-05-08T21:44:33.112 INFO:teuthology.orchestra.run.smithi064.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.3 MB/s | 170 kB 00:00 2024-05-08T21:44:33.129 INFO:teuthology.orchestra.run.smithi064.stdout:(36/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.6 MB/s | 263 kB 00:00 2024-05-08T21:44:33.205 INFO:teuthology.orchestra.run.smithi064.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 196 kB/s | 18 kB 00:00 2024-05-08T21:44:33.342 INFO:teuthology.orchestra.run.smithi186.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 2.2 MB/s | 711 kB 00:00 2024-05-08T21:44:33.476 INFO:teuthology.orchestra.run.smithi186.stdout:(135/146): librados2-18.2.2-1322.gd21e4f96.el8. 3.7 MB/s | 3.7 MB 00:01 2024-05-08T21:44:33.534 INFO:teuthology.orchestra.run.smithi186.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.7 MB/s | 1.5 MB 00:00 2024-05-08T21:44:33.593 INFO:teuthology.orchestra.run.smithi186.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-05-08T21:44:33.685 INFO:teuthology.orchestra.run.smithi186.stdout:(138/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 3.0 MB/s | 3.6 MB 00:01 2024-05-08T21:44:33.727 INFO:teuthology.orchestra.run.smithi186.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-05-08T21:44:33.752 INFO:teuthology.orchestra.run.smithi186.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 221 kB 00:00 2024-05-08T21:44:33.827 INFO:teuthology.orchestra.run.smithi186.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 254 kB 00:00 2024-05-08T21:44:33.869 INFO:teuthology.orchestra.run.smithi186.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.5 MB/s | 180 kB 00:00 2024-05-08T21:44:33.895 INFO:teuthology.orchestra.run.smithi186.stdout:(143/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 237 kB 00:00 2024-05-08T21:44:33.920 INFO:teuthology.orchestra.run.smithi186.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 1.0 MB/s | 99 kB 00:00 2024-05-08T21:44:33.930 INFO:teuthology.orchestra.run.smithi064.stdout:(38/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 7.9 MB/s | 47 MB 00:05 2024-05-08T21:44:34.005 INFO:teuthology.orchestra.run.smithi064.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 350 kB/s | 306 kB 00:00 2024-05-08T21:44:34.089 INFO:teuthology.orchestra.run.smithi064.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 66 kB/s | 59 kB 00:00 2024-05-08T21:44:34.122 INFO:teuthology.orchestra.run.smithi064.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-08T21:44:34.148 INFO:teuthology.orchestra.run.smithi064.stdout:(42/146): libunwind-1.3.1-3.el8.x86_64.rpm 346 kB/s | 75 kB 00:00 2024-05-08T21:44:34.170 INFO:teuthology.orchestra.run.smithi186.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 374 kB/s | 103 kB 00:00 2024-05-08T21:44:34.173 INFO:teuthology.orchestra.run.smithi064.stdout:(43/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 520 kB/s | 44 kB 00:00 2024-05-08T21:44:34.199 INFO:teuthology.orchestra.run.smithi064.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2024-05-08T21:44:34.220 INFO:teuthology.orchestra.run.smithi186.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 7.1 MB/s | 2.5 MB 00:00 2024-05-08T21:44:34.224 INFO:teuthology.orchestra.run.smithi064.stdout:(45/146): python3-cachetools-3.1.1-4.el8.noarch 436 kB/s | 33 kB 00:00 2024-05-08T21:44:34.225 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:44:34.225 INFO:teuthology.orchestra.run.smithi186.stdout:Total 16 MB/s | 215 MB 00:13 2024-05-08T21:44:34.249 INFO:teuthology.orchestra.run.smithi064.stdout:(46/146): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2024-05-08T21:44:34.274 INFO:teuthology.orchestra.run.smithi064.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2024-05-08T21:44:34.308 INFO:teuthology.orchestra.run.smithi064.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2024-05-08T21:44:34.341 INFO:teuthology.orchestra.run.smithi064.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2024-05-08T21:44:34.375 INFO:teuthology.orchestra.run.smithi064.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2024-05-08T21:44:34.401 INFO:teuthology.orchestra.run.smithi064.stdout:(51/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-05-08T21:44:34.442 INFO:teuthology.orchestra.run.smithi064.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 730 kB/s | 49 kB 00:00 2024-05-08T21:44:34.498 INFO:teuthology.orchestra.run.smithi064.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 7.6 MB/s | 1.2 MB 00:00 2024-05-08T21:44:34.526 INFO:teuthology.orchestra.run.smithi064.stdout:(54/146): python3-more-itertools-7.2.0-3.el8.no 468 kB/s | 59 kB 00:00 2024-05-08T21:44:34.552 INFO:teuthology.orchestra.run.smithi064.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 553 kB/s | 60 kB 00:00 2024-05-08T21:44:34.577 INFO:teuthology.orchestra.run.smithi064.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.4 MB/s | 283 kB 00:00 2024-05-08T21:44:34.619 INFO:teuthology.orchestra.run.smithi064.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2024-05-08T21:44:34.644 INFO:teuthology.orchestra.run.smithi064.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2024-05-08T21:44:34.878 INFO:teuthology.orchestra.run.smithi064.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 248 kB/s | 64 kB 00:00 2024-05-08T21:44:34.903 INFO:teuthology.orchestra.run.smithi064.stdout:(60/146): python3-routes-2.4.1-12.el8.noarch.rp 603 kB/s | 196 kB 00:00 2024-05-08T21:44:34.928 INFO:teuthology.orchestra.run.smithi064.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 66 kB/s | 19 kB 00:00 2024-05-08T21:44:34.964 INFO:teuthology.orchestra.run.smithi160.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 1.9 MB/s | 18 MB 00:09 2024-05-08T21:44:34.978 INFO:teuthology.orchestra.run.smithi064.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-05-08T21:44:35.004 INFO:teuthology.orchestra.run.smithi064.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 290 kB/s | 29 kB 00:00 2024-05-08T21:44:35.046 INFO:teuthology.orchestra.run.smithi064.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2024-05-08T21:44:35.099 INFO:teuthology.orchestra.run.smithi160.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.9 MB/s | 261 kB 00:00 2024-05-08T21:44:35.121 INFO:teuthology.orchestra.run.smithi064.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2024-05-08T21:44:35.146 INFO:teuthology.orchestra.run.smithi064.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 613 kB/s | 61 kB 00:00 2024-05-08T21:44:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2024-05-08T21:44:35.180 INFO:teuthology.orchestra.run.smithi064.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2024-05-08T21:44:35.205 INFO:teuthology.orchestra.run.smithi064.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T21:44:35.230 INFO:teuthology.orchestra.run.smithi064.stdout:(70/146): python3-zc-lockfile-2.0-2.el8.noarch. 273 kB/s | 23 kB 00:00 2024-05-08T21:44:35.303 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-05-08T21:44:35.322 INFO:teuthology.orchestra.run.smithi064.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2024-05-08T21:44:35.356 INFO:teuthology.orchestra.run.smithi064.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 119 kB/s | 18 kB 00:00 2024-05-08T21:44:35.440 INFO:teuthology.orchestra.run.smithi064.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 204 kB/s | 43 kB 00:00 2024-05-08T21:44:35.490 INFO:teuthology.orchestra.run.smithi064.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 926 kB/s | 155 kB 00:00 2024-05-08T21:44:35.531 INFO:teuthology.orchestra.run.smithi160.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 72 kB/s | 31 kB 00:00 2024-05-08T21:44:35.532 INFO:teuthology.orchestra.run.smithi064.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 652 kB/s | 109 kB 00:00 2024-05-08T21:44:35.574 INFO:teuthology.orchestra.run.smithi064.stdout:(76/146): python3-requests-oauthlib-1.0.0-1.el8 318 kB/s | 43 kB 00:00 2024-05-08T21:44:35.633 INFO:teuthology.orchestra.run.smithi064.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 131 kB/s | 19 kB 00:00 2024-05-08T21:44:35.658 INFO:teuthology.orchestra.run.smithi064.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 349 kB/s | 44 kB 00:00 2024-05-08T21:44:35.699 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-05-08T21:44:35.699 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-05-08T21:44:35.700 INFO:teuthology.orchestra.run.smithi064.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2024-05-08T21:44:35.750 INFO:teuthology.orchestra.run.smithi064.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2024-05-08T21:44:35.765 INFO:teuthology.orchestra.run.smithi160.stdout:(13/146): ceph-common-18.2.2-1322.gd21e4f96.el8 1.5 MB/s | 22 MB 00:14 2024-05-08T21:44:35.784 INFO:teuthology.orchestra.run.smithi064.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-05-08T21:44:35.865 INFO:teuthology.orchestra.run.smithi160.stdout:(14/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.3 MB/s | 782 kB 00:00 2024-05-08T21:44:35.873 INFO:teuthology.orchestra.run.smithi064.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.1 MB/s | 200 kB 00:00 2024-05-08T21:44:35.890 INFO:teuthology.orchestra.run.smithi160.stdout:(15/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.1 MB/s | 142 kB 00:00 2024-05-08T21:44:35.901 INFO:teuthology.orchestra.run.smithi064.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 320 kB/s | 48 kB 00:00 2024-05-08T21:44:35.932 INFO:teuthology.orchestra.run.smithi160.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 2.0 MB/s | 139 kB 00:00 2024-05-08T21:44:35.976 INFO:teuthology.orchestra.run.smithi064.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 453 kB/s | 87 kB 00:00 2024-05-08T21:44:36.026 INFO:teuthology.orchestra.run.smithi064.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 377 kB/s | 58 kB 00:00 2024-05-08T21:44:36.052 INFO:teuthology.orchestra.run.smithi064.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 256 kB/s | 39 kB 00:00 2024-05-08T21:44:36.074 INFO:teuthology.orchestra.run.smithi160.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.5 MB/s | 474 kB 00:00 2024-05-08T21:44:36.085 INFO:teuthology.orchestra.run.smithi064.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 761 kB/s | 83 kB 00:00 2024-05-08T21:44:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 766 kB/s | 45 kB 00:00 2024-05-08T21:44:36.177 INFO:teuthology.orchestra.run.smithi064.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-05-08T21:44:36.200 INFO:teuthology.orchestra.run.smithi160.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.9 MB/s | 129 kB 00:00 2024-05-08T21:44:36.227 INFO:teuthology.orchestra.run.smithi064.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.1 MB/s | 645 kB 00:00 2024-05-08T21:44:36.300 INFO:teuthology.orchestra.run.smithi160.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.9 MB/s | 197 kB 00:00 2024-05-08T21:44:36.335 INFO:teuthology.orchestra.run.smithi064.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 955 kB/s | 238 kB 00:00 2024-05-08T21:44:36.361 INFO:teuthology.orchestra.run.smithi064.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 461 kB/s | 62 kB 00:00 2024-05-08T21:44:36.386 INFO:teuthology.orchestra.run.smithi064.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 347 kB/s | 73 kB 00:00 2024-05-08T21:44:36.442 INFO:teuthology.orchestra.run.smithi160.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 2.6 MB/s | 370 kB 00:00 2024-05-08T21:44:36.470 INFO:teuthology.orchestra.run.smithi064.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.1 MB/s | 562 kB 00:00 2024-05-08T21:44:36.554 INFO:teuthology.orchestra.run.smithi064.stdout:(94/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-05-08T21:44:36.579 INFO:teuthology.orchestra.run.smithi064.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 245 kB/s | 47 kB 00:00 2024-05-08T21:44:36.584 INFO:teuthology.orchestra.run.smithi160.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 2.5 MB/s | 365 kB 00:00 2024-05-08T21:44:36.629 INFO:teuthology.orchestra.run.smithi064.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.4 MB/s | 559 kB 00:00 2024-05-08T21:44:36.650 INFO:teuthology.orchestra.run.smithi160.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.6 MB/s | 107 kB 00:00 2024-05-08T21:44:36.709 INFO:teuthology.orchestra.run.smithi160.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.6 MB/s | 95 kB 00:00 2024-05-08T21:44:36.713 INFO:teuthology.orchestra.run.smithi064.stdout:(97/146): NetworkManager-initscripts-updown-1.4 908 kB/s | 144 kB 00:00 2024-05-08T21:44:36.747 INFO:teuthology.orchestra.run.smithi064.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 597 kB/s | 100 kB 00:00 2024-05-08T21:44:36.796 INFO:teuthology.orchestra.run.smithi064.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 290 kB/s | 48 kB 00:00 2024-05-08T21:44:36.839 INFO:teuthology.orchestra.run.smithi064.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 159 kB/s | 20 kB 00:00 2024-05-08T21:44:36.864 INFO:teuthology.orchestra.run.smithi064.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 376 kB/s | 44 kB 00:00 2024-05-08T21:44:36.889 INFO:teuthology.orchestra.run.smithi064.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 1.2 MB/s | 110 kB 00:00 2024-05-08T21:44:36.899 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-05-08T21:44:36.910 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-05-08T21:44:36.965 INFO:teuthology.orchestra.run.smithi064.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 211 kB/s | 27 kB 00:00 2024-05-08T21:44:36.990 INFO:teuthology.orchestra.run.smithi064.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2024-05-08T21:44:37.032 INFO:teuthology.orchestra.run.smithi064.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 3.1 MB/s | 457 kB 00:00 2024-05-08T21:44:37.124 INFO:teuthology.orchestra.run.smithi064.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2024-05-08T21:44:37.149 INFO:teuthology.orchestra.run.smithi064.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.0 MB/s | 320 kB 00:00 2024-05-08T21:44:37.199 INFO:teuthology.orchestra.run.smithi064.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2024-05-08T21:44:37.267 INFO:teuthology.orchestra.run.smithi064.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 273 kB/s | 39 kB 00:00 2024-05-08T21:44:37.292 INFO:teuthology.orchestra.run.smithi064.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2024-05-08T21:44:37.317 INFO:teuthology.orchestra.run.smithi064.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 545 kB/s | 64 kB 00:00 2024-05-08T21:44:37.342 INFO:teuthology.orchestra.run.smithi064.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.3 MB/s | 103 kB 00:00 2024-05-08T21:44:37.451 INFO:teuthology.orchestra.run.smithi160.stdout:(25/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 4.3 MB/s | 6.5 MB 00:01 2024-05-08T21:44:37.484 INFO:teuthology.orchestra.run.smithi064.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 3.2 MB/s | 538 kB 00:00 2024-05-08T21:44:37.518 INFO:teuthology.orchestra.run.smithi160.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 2.7 MB/s | 181 kB 00:00 2024-05-08T21:44:37.585 INFO:teuthology.orchestra.run.smithi160.stdout:(27/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 3.9 MB/s | 3.5 MB 00:00 2024-05-08T21:44:37.593 INFO:teuthology.orchestra.run.smithi064.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-05-08T21:44:37.610 INFO:teuthology.orchestra.run.smithi160.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 264 kB/s | 24 kB 00:00 2024-05-08T21:44:37.668 INFO:teuthology.orchestra.run.smithi160.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.6 MB/s | 138 kB 00:00 2024-05-08T21:44:37.676 INFO:teuthology.orchestra.run.smithi064.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-05-08T21:44:37.718 INFO:teuthology.orchestra.run.smithi064.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-05-08T21:44:37.735 INFO:teuthology.orchestra.run.smithi064.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2024-05-08T21:44:37.852 INFO:teuthology.orchestra.run.smithi064.stdout:(118/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-05-08T21:44:37.886 INFO:teuthology.orchestra.run.smithi064.stdout:(119/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2024-05-08T21:44:37.978 INFO:teuthology.orchestra.run.smithi064.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 20 MB/s | 5.2 MB 00:00 2024-05-08T21:44:38.020 INFO:teuthology.orchestra.run.smithi064.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 5.2 MB/s | 714 kB 00:00 2024-05-08T21:44:38.086 INFO:teuthology.orchestra.run.smithi064.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.8 MB/s | 198 kB 00:00 2024-05-08T21:44:38.119 INFO:teuthology.orchestra.run.smithi160.stdout:(30/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 7.1 MB/s | 47 MB 00:06 2024-05-08T21:44:38.195 INFO:teuthology.orchestra.run.smithi160.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 3.2 MB/s | 245 kB 00:00 2024-05-08T21:44:38.204 INFO:teuthology.orchestra.run.smithi064.stdout:(123/146): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-05-08T21:44:38.229 INFO:teuthology.orchestra.run.smithi064.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 382 kB/s | 54 kB 00:00 2024-05-08T21:44:38.245 INFO:teuthology.orchestra.run.smithi160.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 988 kB/s | 49 kB 00:00 2024-05-08T21:44:38.254 INFO:teuthology.orchestra.run.smithi064.stdout:(125/146): jq-1.6-7.el8.x86_64.rpm 864 kB/s | 202 kB 00:00 2024-05-08T21:44:38.295 INFO:teuthology.orchestra.run.smithi160.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 327 kB/s | 16 kB 00:00 2024-05-08T21:44:38.346 INFO:teuthology.orchestra.run.smithi064.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-08T21:44:38.387 INFO:teuthology.orchestra.run.smithi160.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.8 MB/s | 263 kB 00:00 2024-05-08T21:44:38.437 INFO:teuthology.orchestra.run.smithi160.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 3.3 MB/s | 170 kB 00:00 2024-05-08T21:44:38.463 INFO:teuthology.orchestra.run.smithi064.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.2 MB/s | 1.5 MB 00:00 2024-05-08T21:44:38.479 INFO:teuthology.orchestra.run.smithi160.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 436 kB/s | 18 kB 00:00 2024-05-08T21:44:38.522 INFO:teuthology.orchestra.run.smithi064.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.5 MB/s | 812 kB 00:00 2024-05-08T21:44:38.688 INFO:teuthology.orchestra.run.smithi160.stdout:(37/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 3.3 MB/s | 3.5 MB 00:01 2024-05-08T21:44:38.722 INFO:teuthology.orchestra.run.smithi064.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2024-05-08T21:44:38.789 INFO:teuthology.orchestra.run.smithi064.stdout:(130/146): postgresql-server-10.23-4.module+el8 16 MB/s | 5.1 MB 00:00 2024-05-08T21:44:38.964 INFO:teuthology.orchestra.run.smithi064.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2024-05-08T21:44:38.971 INFO:teuthology.orchestra.run.smithi160.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 623 kB/s | 306 kB 00:00 2024-05-08T21:44:39.013 INFO:teuthology.orchestra.run.smithi160.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 181 kB/s | 59 kB 00:00 2024-05-08T21:44:39.248 INFO:teuthology.orchestra.run.smithi064.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.0 MB/s | 303 kB 00:00 2024-05-08T21:44:39.288 INFO:teuthology.orchestra.run.smithi160.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 237 kB/s | 75 kB 00:00 2024-05-08T21:44:39.306 INFO:teuthology.orchestra.run.smithi064.stdout:(133/146): librados2-18.2.2-1322.gd21e4f96.el8. 4.7 MB/s | 3.7 MB 00:00 2024-05-08T21:44:39.365 INFO:teuthology.orchestra.run.smithi064.stdout:(134/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 5.5 MB/s | 3.6 MB 00:00 2024-05-08T21:44:39.415 INFO:teuthology.orchestra.run.smithi064.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2024-05-08T21:44:39.474 INFO:teuthology.orchestra.run.smithi064.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.2 MB/s | 711 kB 00:00 2024-05-08T21:44:39.564 INFO:teuthology.orchestra.run.smithi160.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 865 kB/s | 476 kB 00:00 2024-05-08T21:44:39.574 INFO:teuthology.orchestra.run.smithi064.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.0 MB/s | 1.5 MB 00:00 2024-05-08T21:44:39.608 INFO:teuthology.orchestra.run.smithi064.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.7 MB/s | 333 kB 00:00 2024-05-08T21:44:39.647 INFO:teuthology.orchestra.run.smithi160.stdout:(42/146): ceph-mgr-diskprediction-local-18.2.2- 3.7 MB/s | 7.4 MB 00:01 2024-05-08T21:44:39.673 INFO:teuthology.orchestra.run.smithi160.stdout:(43/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 113 kB/s | 44 kB 00:00 2024-05-08T21:44:39.675 INFO:teuthology.orchestra.run.smithi064.stdout:(139/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 3.7 MB/s | 254 kB 00:00 2024-05-08T21:44:39.725 INFO:teuthology.orchestra.run.smithi064.stdout:(140/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.3 MB/s | 1.1 MB 00:00 2024-05-08T21:44:39.792 INFO:teuthology.orchestra.run.smithi064.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.0 MB/s | 237 kB 00:00 2024-05-08T21:44:39.817 INFO:teuthology.orchestra.run.smithi064.stdout:(142/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 910 kB/s | 221 kB 00:00 2024-05-08T21:44:39.890 INFO:teuthology.orchestra.run.smithi160.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 136 kB/s | 33 kB 00:00 2024-05-08T21:44:39.892 INFO:teuthology.orchestra.run.smithi064.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 180 kB 00:00 2024-05-08T21:44:39.923 INFO:teuthology.orchestra.run.smithi160.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 65 kB/s | 16 kB 00:00 2024-05-08T21:44:39.965 INFO:teuthology.orchestra.run.smithi160.stdout:(46/146): python3-beautifulsoup4-4.6.3-2.el8.1. 462 kB/s | 185 kB 00:00 2024-05-08T21:44:40.000 INFO:teuthology.orchestra.run.smithi064.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 14 MB/s | 2.5 MB 00:00 2024-05-08T21:44:40.026 INFO:teuthology.orchestra.run.smithi064.stdout:(145/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 424 kB/s | 99 kB 00:00 2024-05-08T21:44:40.051 INFO:teuthology.orchestra.run.smithi064.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 648 kB/s | 103 kB 00:00 2024-05-08T21:44:40.055 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:44:40.055 INFO:teuthology.orchestra.run.smithi064.stdout:Total 14 MB/s | 215 MB 00:15 2024-05-08T21:44:40.290 INFO:teuthology.orchestra.run.smithi160.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 250 kB/s | 81 kB 00:00 2024-05-08T21:44:40.365 INFO:teuthology.orchestra.run.smithi160.stdout:(48/146): python3-cheroot-8.5.2-1.el8.noarch.rp 363 kB/s | 173 kB 00:00 2024-05-08T21:44:40.474 INFO:teuthology.orchestra.run.smithi160.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 698 kB/s | 384 kB 00:00 2024-05-08T21:44:40.500 INFO:teuthology.orchestra.run.smithi160.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 53 kB/s | 11 kB 00:00 2024-05-08T21:44:40.650 INFO:teuthology.orchestra.run.smithi160.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 62 kB/s | 18 kB 00:00 2024-05-08T21:44:40.879 INFO:teuthology.orchestra.run.smithi160.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 128 kB/s | 49 kB 00:00 2024-05-08T21:44:40.983 INFO:teuthology.orchestra.run.smithi160.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 176 kB/s | 59 kB 00:00 2024-05-08T21:44:41.093 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-05-08T21:44:41.192 INFO:teuthology.orchestra.run.smithi160.stdout:(54/146): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2024-05-08T21:44:41.234 INFO:teuthology.orchestra.run.smithi160.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 170 kB/s | 60 kB 00:00 2024-05-08T21:44:41.459 INFO:teuthology.orchestra.run.smithi160.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 58 kB/s | 16 kB 00:00 2024-05-08T21:44:41.485 INFO:teuthology.orchestra.run.smithi160.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 565 kB/s | 283 kB 00:00 2024-05-08T21:44:41.510 INFO:teuthology.orchestra.run.smithi160.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 122 kB/s | 34 kB 00:00 2024-05-08T21:44:41.519 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-05-08T21:44:41.520 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-05-08T21:44:41.760 INFO:teuthology.orchestra.run.smithi160.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 75 kB/s | 19 kB 00:00 2024-05-08T21:44:41.794 INFO:teuthology.orchestra.run.smithi160.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 208 kB/s | 64 kB 00:00 2024-05-08T21:44:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:(61/146): python3-routes-2.4.1-12.el8.noarch.rp 523 kB/s | 196 kB 00:00 2024-05-08T21:44:42.028 INFO:teuthology.orchestra.run.smithi160.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 89 kB/s | 24 kB 00:00 2024-05-08T21:44:42.044 INFO:teuthology.orchestra.run.smithi160.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 117 kB/s | 29 kB 00:00 2024-05-08T21:44:42.078 INFO:teuthology.orchestra.run.smithi160.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 114 kB/s | 27 kB 00:00 2024-05-08T21:44:42.412 INFO:teuthology.orchestra.run.smithi160.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 184 kB/s | 61 kB 00:00 2024-05-08T21:44:42.432 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-05-08T21:44:42.504 INFO:teuthology.orchestra.run.smithi160.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 547 kB/s | 251 kB 00:00 2024-05-08T21:44:42.529 INFO:teuthology.orchestra.run.smithi160.stdout:(67/146): python3-waitress-1.4.3-1.el8.noarch.r 510 kB/s | 255 kB 00:00 2024-05-08T21:44:42.567 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:44:42.746 INFO:teuthology.orchestra.run.smithi160.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 256 kB/s | 85 kB 00:00 2024-05-08T21:44:42.771 INFO:teuthology.orchestra.run.smithi160.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 86 kB/s | 23 kB 00:00 2024-05-08T21:44:42.795 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-05-08T21:44:42.796 INFO:teuthology.orchestra.run.smithi160.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-05-08T21:44:42.806 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-05-08T21:44:42.829 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:44:42.930 INFO:teuthology.orchestra.run.smithi160.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 113 kB/s | 18 kB 00:00 2024-05-08T21:44:43.006 INFO:teuthology.orchestra.run.smithi160.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 209 kB/s | 43 kB 00:00 2024-05-08T21:44:43.031 INFO:teuthology.orchestra.run.smithi160.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2024-05-08T21:44:43.044 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:44:43.156 INFO:teuthology.orchestra.run.smithi160.stdout:(74/146): python3-requests-oauthlib-1.0.0-1.el8 341 kB/s | 43 kB 00:00 2024-05-08T21:44:43.181 INFO:teuthology.orchestra.run.smithi160.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 623 kB/s | 109 kB 00:00 2024-05-08T21:44:43.202 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:44:43.273 INFO:teuthology.orchestra.run.smithi160.stdout:(76/146): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2024-05-08T21:44:43.298 INFO:teuthology.orchestra.run.smithi160.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 375 kB/s | 44 kB 00:00 2024-05-08T21:44:43.349 INFO:teuthology.orchestra.run.smithi160.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 97 kB/s | 19 kB 00:00 2024-05-08T21:44:43.365 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:44:43.441 INFO:teuthology.orchestra.run.smithi160.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-05-08T21:44:43.463 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:44:43.482 INFO:teuthology.orchestra.run.smithi160.stdout:(80/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2024-05-08T21:44:43.508 INFO:teuthology.orchestra.run.smithi160.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-05-08T21:44:43.583 INFO:teuthology.orchestra.run.smithi160.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2024-05-08T21:44:43.600 INFO:teuthology.orchestra.run.smithi160.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 410 kB/s | 48 kB 00:00 2024-05-08T21:44:43.623 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:44:43.625 INFO:teuthology.orchestra.run.smithi160.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 743 kB/s | 87 kB 00:00 2024-05-08T21:44:43.674 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:44:43.709 INFO:teuthology.orchestra.run.smithi160.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 355 kB/s | 39 kB 00:00 2024-05-08T21:44:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:(86/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 405 kB/s | 58 kB 00:00 2024-05-08T21:44:43.751 INFO:teuthology.orchestra.run.smithi160.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 662 kB/s | 83 kB 00:00 2024-05-08T21:44:43.818 INFO:teuthology.orchestra.run.smithi160.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.8 MB/s | 645 kB 00:00 2024-05-08T21:44:43.852 INFO:teuthology.orchestra.run.smithi160.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-05-08T21:44:43.927 INFO:teuthology.orchestra.run.smithi160.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-05-08T21:44:43.981 INFO:teuthology.orchestra.run.smithi160.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 479 kB/s | 62 kB 00:00 2024-05-08T21:44:44.003 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:44:44.077 INFO:teuthology.orchestra.run.smithi160.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 280 kB/s | 73 kB 00:00 2024-05-08T21:44:44.111 INFO:teuthology.orchestra.run.smithi160.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.0 MB/s | 562 kB 00:00 2024-05-08T21:44:44.228 INFO:teuthology.orchestra.run.smithi160.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 313 kB/s | 47 kB 00:00 2024-05-08T21:44:44.311 INFO:teuthology.orchestra.run.smithi160.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 7.0 MB/s | 2.3 MB 00:00 2024-05-08T21:44:44.353 INFO:teuthology.orchestra.run.smithi160.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 2.3 MB/s | 559 kB 00:00 2024-05-08T21:44:44.379 INFO:teuthology.orchestra.run.smithi160.stdout:(97/146): NetworkManager-initscripts-updown-1.4 960 kB/s | 144 kB 00:00 2024-05-08T21:44:44.454 INFO:teuthology.orchestra.run.smithi160.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 703 kB/s | 100 kB 00:00 2024-05-08T21:44:44.464 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T21:44:44.496 INFO:teuthology.orchestra.run.smithi160.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 339 kB/s | 48 kB 00:00 2024-05-08T21:44:44.562 INFO:teuthology.orchestra.run.smithi160.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 109 kB/s | 20 kB 00:00 2024-05-08T21:44:44.605 INFO:teuthology.orchestra.run.smithi160.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 292 kB/s | 44 kB 00:00 2024-05-08T21:44:44.630 INFO:teuthology.orchestra.run.smithi160.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 823 kB/s | 110 kB 00:00 2024-05-08T21:44:44.665 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:44:44.680 INFO:teuthology.orchestra.run.smithi160.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 225 kB/s | 27 kB 00:00 2024-05-08T21:44:44.706 INFO:teuthology.orchestra.run.smithi160.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.2 MB/s | 126 kB 00:00 2024-05-08T21:44:44.740 INFO:teuthology.orchestra.run.smithi160.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 4.1 MB/s | 457 kB 00:00 2024-05-08T21:44:44.773 INFO:teuthology.orchestra.run.smithi160.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2024-05-08T21:44:44.791 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:44:44.833 INFO:teuthology.orchestra.run.smithi160.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 4.4 MB/s | 688 kB 00:00 2024-05-08T21:44:44.898 INFO:teuthology.orchestra.run.smithi160.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2024-05-08T21:44:44.924 INFO:teuthology.orchestra.run.smithi160.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 258 kB/s | 39 kB 00:00 2024-05-08T21:44:44.973 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:44:44.974 INFO:teuthology.orchestra.run.smithi160.stdout:(110/146): uuid-1.6.2-43.el8.x86_64.rpm 851 kB/s | 64 kB 00:00 2024-05-08T21:44:45.049 INFO:teuthology.orchestra.run.smithi160.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 826 kB/s | 103 kB 00:00 2024-05-08T21:44:45.074 INFO:teuthology.orchestra.run.smithi160.stdout:(112/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2024-05-08T21:44:45.208 INFO:teuthology.orchestra.run.smithi160.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 3.3 MB/s | 538 kB 00:00 2024-05-08T21:44:45.325 INFO:teuthology.orchestra.run.smithi160.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2024-05-08T21:44:45.371 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:44:45.434 INFO:teuthology.orchestra.run.smithi160.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 10 MB/s | 3.7 MB 00:00 2024-05-08T21:44:45.459 INFO:teuthology.orchestra.run.smithi160.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-05-08T21:44:45.493 INFO:teuthology.orchestra.run.smithi160.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-05-08T21:44:45.548 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:44:45.585 INFO:teuthology.orchestra.run.smithi160.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2024-05-08T21:44:45.626 INFO:teuthology.orchestra.run.smithi160.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-05-08T21:44:45.768 INFO:teuthology.orchestra.run.smithi160.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 17 MB/s | 5.2 MB 00:00 2024-05-08T21:44:45.810 INFO:teuthology.orchestra.run.smithi160.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 3.8 MB/s | 714 kB 00:00 2024-05-08T21:44:45.942 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:44:46.094 INFO:teuthology.orchestra.run.smithi160.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 9.9 MB/s | 5.0 MB 00:00 2024-05-08T21:44:46.098 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:44:46.232 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:44:46.327 INFO:teuthology.orchestra.run.smithi160.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 354 kB/s | 198 kB 00:00 2024-05-08T21:44:46.353 INFO:teuthology.orchestra.run.smithi160.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 373 kB/s | 202 kB 00:00 2024-05-08T21:44:46.369 INFO:teuthology.orchestra.run.smithi160.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 197 kB/s | 54 kB 00:00 2024-05-08T21:44:46.442 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:44:46.445 INFO:teuthology.orchestra.run.smithi160.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-08T21:44:46.553 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:44:46.554 INFO:teuthology.orchestra.run.smithi160.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 8.2 MB/s | 1.5 MB 00:00 2024-05-08T21:44:46.629 INFO:teuthology.orchestra.run.smithi160.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.3 MB/s | 812 kB 00:00 2024-05-08T21:44:46.821 INFO:teuthology.orchestra.run.smithi160.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 28 MB/s | 14 MB 00:00 2024-05-08T21:44:46.888 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:44:46.921 INFO:teuthology.orchestra.run.smithi160.stdout:(130/146): postgresql-server-10.23-4.module+el8 14 MB/s | 5.1 MB 00:00 2024-05-08T21:44:47.063 INFO:teuthology.orchestra.run.smithi160.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T21:44:47.106 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:44:47.196 INFO:teuthology.orchestra.run.smithi160.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.2 MB/s | 303 kB 00:00 2024-05-08T21:44:47.211 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:44:47.280 INFO:teuthology.orchestra.run.smithi160.stdout:(133/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 7.8 MB/s | 3.6 MB 00:00 2024-05-08T21:44:47.388 INFO:teuthology.orchestra.run.smithi160.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.3 MB/s | 854 kB 00:00 2024-05-08T21:44:47.429 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:44:47.447 INFO:teuthology.orchestra.run.smithi160.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.1 MB/s | 711 kB 00:00 2024-05-08T21:44:47.498 INFO:teuthology.orchestra.run.smithi160.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 14 MB/s | 1.5 MB 00:00 2024-05-08T21:44:47.565 INFO:teuthology.orchestra.run.smithi160.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-05-08T21:44:47.623 INFO:teuthology.orchestra.run.smithi160.stdout:(138/146): librados2-18.2.2-1322.gd21e4f96.el8. 3.7 MB/s | 3.7 MB 00:00 2024-05-08T21:44:47.684 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:44:47.705 INFO:teuthology.orchestra.run.smithi160.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 221 kB 00:00 2024-05-08T21:44:47.749 INFO:teuthology.orchestra.run.smithi160.stdout:(140/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.3 MB/s | 1.1 MB 00:00 2024-05-08T21:44:47.774 INFO:teuthology.orchestra.run.smithi160.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 254 kB 00:00 2024-05-08T21:44:47.841 INFO:teuthology.orchestra.run.smithi160.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 237 kB 00:00 2024-05-08T21:44:47.893 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:44:47.899 INFO:teuthology.orchestra.run.smithi160.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-08T21:44:47.925 INFO:teuthology.orchestra.run.smithi160.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 658 kB/s | 99 kB 00:00 2024-05-08T21:44:47.999 INFO:teuthology.orchestra.run.smithi160.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 1.0 MB/s | 103 kB 00:00 2024-05-08T21:44:48.067 INFO:teuthology.orchestra.run.smithi160.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 11 MB/s | 2.5 MB 00:00 2024-05-08T21:44:48.069 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:44:48.069 INFO:teuthology.orchestra.run.smithi160.stdout:Total 8.0 MB/s | 215 MB 00:26 2024-05-08T21:44:48.253 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:44:48.526 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:44:48.568 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-05-08T21:44:48.690 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:44:48.710 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:44:48.780 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:44:48.973 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:44:48.974 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:44:49.052 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:44:49.130 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-08T21:44:49.201 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:44:49.217 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T21:44:49.375 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:44:49.506 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T21:44:49.522 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:44:49.525 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-08T21:44:49.525 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-08T21:44:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:44:49.763 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:44:49.836 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T21:44:49.898 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:44:49.981 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:44:50.110 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:44:50.272 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:44:50.385 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:44:50.573 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T21:44:50.728 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-08T21:44:50.739 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-08T21:44:50.771 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:44:50.876 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:44:50.906 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:44:51.064 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:44:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:44:51.507 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:44:51.654 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:44:51.687 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:44:51.839 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:44:52.101 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:44:52.262 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:44:52.394 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:44:52.528 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:44:52.598 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:44:52.679 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:44:52.812 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T21:44:53.012 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T21:44:53.026 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:44:53.245 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:44:53.250 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:44:53.377 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:44:53.437 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:44:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:44:53.575 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:44:53.820 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:44:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:44:53.991 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:44:54.043 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:44:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:44:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:44:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:44:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:44:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:44:55.068 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:44:55.258 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T21:44:55.518 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T21:44:55.734 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T21:44:55.802 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:44:55.870 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:44:56.005 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:44:56.138 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T21:44:56.186 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:44:56.512 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-08T21:44:56.674 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:44:56.851 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:44:56.986 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:44:57.037 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:44:57.114 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T21:44:57.224 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:44:57.422 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:44:57.436 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T21:44:57.593 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:44:57.690 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:44:57.692 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:44:57.708 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T21:44:57.852 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:44:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:44:57.924 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:44:57.954 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:44:58.123 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:44:58.157 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:44:58.342 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T21:44:58.361 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:44:58.537 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:44:58.552 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:44:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:44:58.651 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:44:58.730 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:44:58.803 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:44:58.843 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:44:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T21:44:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T21:44:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:44:59.112 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:44:59.244 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:44:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:44:59.334 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:44:59.460 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:44:59.468 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:44:59.549 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:44:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:44:59.755 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:44:59.832 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:44:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:45:00.029 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:45:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:45:00.137 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:45:00.192 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:45:00.346 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:45:00.441 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:45:00.455 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:45:00.481 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:45:00.665 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-08T21:45:00.765 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:45:00.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-08T21:45:00.960 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-08T21:45:01.026 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:45:01.115 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:45:01.223 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-7.el8.x86_64 51/162 2024-05-08T21:45:01.324 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:45:01.480 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-8.el8.noarch 52/162 2024-05-08T21:45:01.620 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:45:01.655 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 53/162 2024-05-08T21:45:01.729 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 53/162 2024-05-08T21:45:01.821 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:45:01.919 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 54/162 2024-05-08T21:45:01.945 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:45:02.154 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 55/162 2024-05-08T21:45:02.204 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:45:02.275 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T21:45:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 56/162 2024-05-08T21:45:02.447 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:45:02.500 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-08T21:45:02.610 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:45:02.700 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:45:02.728 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-08T21:45:02.876 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:45:02.926 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-08T21:45:02.947 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:45:03.116 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-08T21:45:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T21:45:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T21:45:03.325 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-08T21:45:03.426 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T21:45:03.608 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-08T21:45:03.639 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T21:45:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T21:45:03.817 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:45:03.859 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-08T21:45:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T21:45:03.956 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:45:04.084 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-08T21:45:04.101 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:45:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:45:04.233 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-08T21:45:04.357 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:45:04.473 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-08T21:45:04.534 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:45:04.709 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:45:04.725 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:45:04.820 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-08T21:45:04.894 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:45:04.901 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:45:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:45:05.186 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:45:05.255 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:45:05.461 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:45:05.507 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:45:05.550 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-08T21:45:05.664 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:45:05.692 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:45:05.827 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:45:05.888 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-08T21:45:06.086 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-08T21:45:06.263 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-08T21:45:06.305 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:45:06.376 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:45:06.541 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-08T21:45:06.645 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:45:06.665 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T21:45:06.672 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:45:06.774 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-08T21:45:06.844 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T21:45:06.889 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T21:45:06.918 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T21:45:07.012 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-08T21:45:07.152 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:45:07.178 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T21:45:07.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-08T21:45:07.365 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:45:07.378 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T21:45:07.416 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-08T21:45:07.503 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:45:07.535 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T21:45:07.603 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T21:45:07.688 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-08T21:45:07.760 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T21:45:07.779 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:45:07.861 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:45:07.943 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T21:45:07.996 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:45:08.045 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:45:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T21:45:08.297 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-08T21:45:08.346 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:45:08.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-08T21:45:08.559 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:45:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-08T21:45:08.741 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:45:08.892 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-08T21:45:08.938 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:45:09.073 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-08T21:45:09.133 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:45:09.249 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-08T21:45:09.339 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:45:09.558 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-08T21:45:09.565 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:45:09.730 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:45:09.748 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:45:09.833 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-08T21:45:09.900 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:45:10.057 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-08T21:45:10.075 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T21:45:10.165 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:45:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-08T21:45:10.365 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:45:10.458 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-08T21:45:10.525 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:45:10.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-08T21:45:10.806 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:45:11.027 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:45:11.049 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T21:45:11.065 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-08T21:45:11.219 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:45:11.386 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-08T21:45:11.397 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T21:45:11.502 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:45:11.610 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-08T21:45:11.627 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T21:45:11.779 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-08T21:45:11.861 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:45:12.018 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-08T21:45:12.102 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:45:12.274 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-08T21:45:12.305 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:45:12.449 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:45:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-08T21:45:12.582 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:45:12.641 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:45:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-08T21:45:12.705 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:45:12.857 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:45:12.887 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-08T21:45:12.933 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:45:12.999 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:45:13.069 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:45:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-08T21:45:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:45:13.302 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:45:13.379 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:45:13.470 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:45:13.515 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:45:13.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:45:13.700 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:45:13.778 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:45:13.852 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:45:14.026 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:45:14.114 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:45:14.190 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:45:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:45:14.442 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:45:14.470 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:45:14.568 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:45:14.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:45:14.694 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T21:45:14.758 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:45:14.762 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T21:45:14.777 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:45:14.853 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:45:14.984 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:45:15.010 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T21:45:15.069 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:45:15.246 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:45:15.258 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T21:45:15.391 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:45:15.441 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T21:45:15.497 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T21:45:15.617 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:45:15.698 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T21:45:15.848 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T21:45:15.872 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:45:16.084 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T21:45:16.292 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-08T21:45:16.465 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:45:16.509 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-08T21:45:16.714 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-08T21:45:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:45:16.913 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-08T21:45:16.967 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:45:17.119 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-08T21:45:17.176 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:45:17.295 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-08T21:45:17.388 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:45:17.587 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-08T21:45:17.870 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-08T21:45:18.104 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-08T21:45:18.295 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-08T21:45:18.534 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-08T21:45:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:45:18.840 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:45:18.862 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-08T21:45:19.005 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:45:19.194 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-08T21:45:19.277 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:19.319 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:45:19.506 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-08T21:45:19.593 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:45:19.740 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-08T21:45:19.786 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:45:19.869 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:19.950 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-08T21:45:20.149 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:45:20.177 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-08T21:45:20.263 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:45:20.411 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-08T21:45:20.418 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:45:20.589 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:45:20.641 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-08T21:45:20.767 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:45:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-08T21:45:20.844 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:45:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:45:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-08T21:45:21.324 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-08T21:45:21.472 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:45:21.696 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:45:22.001 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:45:22.230 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:45:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:45:22.669 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:45:22.890 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:45:23.102 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:45:23.352 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:45:23.550 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:45:23.729 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:45:23.977 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:45:24.202 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:45:24.345 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:45:24.618 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:45:24.872 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:45:25.097 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:45:25.288 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:25.347 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:45:25.508 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:26.139 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:26.444 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:45:26.454 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:45:26.531 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:45:26.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:45:26.776 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:45:26.996 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:45:27.271 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:45:28.082 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:45:28.258 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:45:29.025 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:45:29.214 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:45:29.278 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:45:29.489 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:45:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:45:34.630 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:45:34.714 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:45:34.830 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:45:40.049 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:45:40.049 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2024-05-08T21:45:40.050 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:45:40.416 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:45:42.059 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:45:42.237 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:45:42.456 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:45:42.607 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:45:43.083 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:45:43.240 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:45:43.258 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:45:45.146 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:45:45.306 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:45:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:45:45.691 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:45:45.920 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:45:46.953 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:45:47.135 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:45:47.135 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-05-08T21:45:47.135 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /run 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-05-08T21:45:47.136 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:45:47.211 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:45:47.497 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:45:48.180 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:45:48.381 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:45:48.475 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:45:48.824 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:45:49.011 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:45:49.050 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:45:49.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:45:49.226 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:45:49.244 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:45:49.416 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:45:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:45:49.763 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:45:50.274 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:45:50.413 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:45:50.431 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:45:50.480 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:45:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:45:50.687 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:45:50.881 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:45:50.924 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:45:51.097 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:45:51.445 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:45:51.631 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:45:52.513 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:45:52.669 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:45:52.795 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:45:53.040 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:45:53.260 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:45:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /run 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-05-08T21:45:54.430 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-05-08T21:45:54.431 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2024-05-08T21:45:54.431 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:45:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:45:54.819 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:45:55.605 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:45:55.779 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:45:55.880 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:45:56.098 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:45:56.262 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:45:56.415 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:45:56.456 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:45:56.458 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:45:56.629 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:45:56.641 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:45:56.643 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:45:56.861 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:45:56.997 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:45:57.043 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:45:57.309 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T21:45:57.519 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:45:57.571 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T21:45:57.702 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:45:57.719 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:45:57.940 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T21:45:57.993 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:45:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:45:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:45:58.282 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T21:45:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:45:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:45:58.550 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:45:58.563 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:45:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:45:58.790 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:45:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:45:58.975 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:45:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:45:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:45:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:45:59.821 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:45:59.924 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:46:00.153 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:46:00.457 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:46:01.458 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:01.488 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:46:01.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:01.751 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:46:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:02.197 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:46:02.503 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:46:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:46:02.741 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:46:02.913 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:46:02.933 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:46:03.029 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:46:03.128 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:46:03.404 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:46:03.439 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:46:03.592 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:46:03.631 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:46:03.722 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:46:03.800 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:46:03.812 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:46:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:46:03.945 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:46:04.021 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:46:04.220 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:46:04.233 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:46:04.387 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:46:04.395 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:46:04.581 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T21:46:04.737 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:46:04.885 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:46:04.922 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:46:05.060 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:46:05.107 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:46:05.107 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:46:05.191 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T21:46:05.259 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:46:05.294 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:46:05.398 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:46:05.398 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:46:05.441 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T21:46:05.525 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:46:05.568 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:46:05.649 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:46:05.649 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:46:05.775 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:46:05.855 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T21:46:05.918 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:46:06.113 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:46:06.121 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:46:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T21:46:06.289 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:46:06.342 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:46:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:46:06.389 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:46:06.547 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:46:06.547 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:06.614 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:46:06.774 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:46:06.797 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:06.798 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:46:06.970 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:46:07.056 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:46:07.056 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:07.314 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:07.314 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:46:07.556 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:46:07.556 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:07.814 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:07.991 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:46:07.991 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:46:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:46:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:46:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:46:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:46:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:46:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:46:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-08T21:46:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-08T21:46:08.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 110/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/162 2024-05-08T21:46:08.921 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.11-1.el8.x86_64 122/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-7.el8.x86_64 123/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 124/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 125/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 126/162 2024-05-08T21:46:08.922 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 127/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 128/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 129/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:46:08.923 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:46:08.924 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:08.925 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:46:09.418 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:09.694 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:09.748 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:46:10.469 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:46:10.723 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:46:10.796 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:46:10.878 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:46:11.094 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:46:11.344 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:46:11.641 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:46:11.696 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:46:11.818 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:46:11.900 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:46:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T21:46:12.101 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:46:12.116 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:46:12.248 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T21:46:12.293 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:46:12.303 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:46:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T21:46:12.623 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T21:46:12.643 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:46:12.804 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:46:12.976 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T21:46:12.981 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:46:12.981 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:46:13.213 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:46:13.214 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:46:13.227 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:46:13.241 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:46:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:46:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:46:13.451 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:46:13.619 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:46:13.667 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:46:13.816 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:46:13.895 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:46:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:46:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:46:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:14.488 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:14.488 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:46:14.738 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:46:14.738 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:14.963 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:14.963 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:46:15.237 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:46:15.238 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:15.454 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:15.604 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:46:15.605 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:46:16.095 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:16.390 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:16.431 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:46:16.484 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:46:16.485 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:46:16.486 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:46:16.498 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:46:16.499 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:46:16.500 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-08T21:46:16.501 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-08T21:46:16.502 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-08T21:46:16.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T21:46:16.504 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T21:46:16.505 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:46:16.506 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:46:16.507 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:16.508 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:46:16.870 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:46:17.280 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:46:17.515 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:46:17.715 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:46:17.922 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:46:18.232 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:46:18.524 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:46:18.739 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:46:18.816 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:46:19.039 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:46:19.052 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:46:19.248 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:46:19.256 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:46:19.441 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T21:46:19.585 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:46:19.774 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:46:19.935 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:46:19.935 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:46:20.209 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:46:20.209 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:46:20.443 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:46:20.443 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:46:20.695 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:46:20.949 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:46:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:46:21.366 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:46:21.367 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:21.608 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:21.608 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:46:21.850 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:46:21.850 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:22.083 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:22.083 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:46:22.350 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:46:22.350 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:22.608 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:22.741 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:46:22.742 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:46:22.940 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:46:22.940 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:46:23.648 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:46:23.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:46:23.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:46:23.651 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:46:23.653 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:46:23.654 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-08T21:46:23.655 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-08T21:46:23.656 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T21:46:23.657 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T21:46:23.658 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:46:23.659 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:46:23.660 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:46:23.661 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:46:24.569 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.570 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.571 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:46:24.573 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.575 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:46:24.575 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.575 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:46:24.575 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:46:24.575 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:46:24.575 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:46:24.576 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:46:24.577 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:46:24.578 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:46:24.579 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:46:24.580 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:46:24.581 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.582 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:46:24.583 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:46:24.584 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-05-08T21:46:24.829 DEBUG:teuthology.parallel:result is None 2024-05-08T21:46:30.700 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:46:30.700 INFO:teuthology.orchestra.run.smithi064.stdout:Installed products updated. 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.534 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.535 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:46:31.536 INFO:teuthology.orchestra.run.smithi064.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:46:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:46:31.538 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:46:31.540 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:46:31.541 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:46:31.542 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:46:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:46:31.544 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:46:31.545 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:46:31.545 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-08T21:46:31.759 DEBUG:teuthology.parallel:result is None 2024-05-08T21:46:34.305 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:46:34.306 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2024-05-08T21:46:34.916 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:34.917 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:34.918 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.918 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.918 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:46:34.918 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:46:34.918 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.919 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:46:34.920 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:46:34.921 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:46:34.922 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:46:34.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:46:34.924 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:46:34.925 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:46:34.926 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.927 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:46:34.928 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:46:34.929 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:46:34.929 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:46:34.929 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:46:34.929 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:46:34.930 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:46:34.930 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T21:46:34.930 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-08T21:46:35.141 DEBUG:teuthology.parallel:result is None 2024-05-08T21:46:35.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:46:35.289 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:46:35.319 INFO:teuthology.orchestra.run.smithi064.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:46:35.319 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:46:35.319 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:46:35.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:46:35.468 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:46:35.493 INFO:teuthology.orchestra.run.smithi160.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:46:35.493 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:46:35.493 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:46:35.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:46:35.714 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:46:35.743 INFO:teuthology.orchestra.run.smithi186.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:46:35.743 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:46:35.743 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:46:35.745 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T21:46:35.745 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:46:35.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:46:35.798 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:46:35.798 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:46:35.843 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:46:35.843 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:46:35.893 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T21:46:35.893 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:46:35.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:46:35.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:46:35.999 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:46:36.000 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:46:36.029 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:46:36.099 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:46:36.099 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:46:36.132 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:46:36.202 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T21:46:36.203 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:46:36.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:46:36.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:46:36.308 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:46:36.308 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:46:36.339 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:46:36.408 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:46:36.408 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:46:36.438 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:46:36.508 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T21:46:36.508 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:46:36.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:46:36.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:46:36.614 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:46:36.615 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:46:36.646 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:46:36.715 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:46:36.715 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:46:36.745 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:46:36.814 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T21:46:36.952 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2}} 2024-05-08T21:46:36.953 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:46:36.953 INFO:tasks.cephadm:Cluster fsid is 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:46:36.953 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T21:46:36.953 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.b': '172.21.15.160', 'mon.c': '172.21.15.186'} 2024-05-08T21:46:36.953 INFO:tasks.cephadm:First mon is mon.a on smithi064 2024-05-08T21:46:36.954 INFO:tasks.cephadm:First mgr is x 2024-05-08T21:46:36.954 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T21:46:36.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-05-08T21:46:36.988 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-05-08T21:46:37.020 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-05-08T21:46:37.050 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T21:46:37.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:46:37.355 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T21:46:37.502 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:46:37.504 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T21:46:37.504 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:38.754 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:46 /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:38.755 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:39.777 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:46 /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:39.778 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:40.659 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:46 /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:40.660 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:40.682 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:40.704 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:46:40.742 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T21:46:40.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:46:40.745 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:46:40.747 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:46:41.697 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:46:42.136 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:46:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:47:18.480 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-08T21:47:18.480 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:47:18.481 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:47:18.481 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-05-08T21:47:18.481 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:47:18.481 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-05-08T21:47:18.481 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-08T21:47:21.075 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:47:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:47:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:47:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-05-08T21:47:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:47:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T21:47:21.076 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:47:22.364 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-08T21:47:22.365 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:47:22.365 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:47:22.365 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-05-08T21:47:22.365 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:47:22.365 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-05-08T21:47:22.365 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-08T21:47:22.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-05-08T21:47:22.422 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-05-08T21:47:22.458 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-05-08T21:47:22.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-05-08T21:47:22.522 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-05-08T21:47:22.552 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-05-08T21:47:22.579 INFO:tasks.cephadm:Writing seed config... 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T21:47:22.580 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T21:47:22.581 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-08T21:47:22.582 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T21:47:22.583 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-08T21:47:22.584 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:47:22.584 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T21:47:22.604 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 71e641de-0d84-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T21:47:22.605 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T21:47:22.647 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T21:47:22.680 INFO:journalctl@ceph.mon.a.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:47:22.680 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T21:47:22.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:47:22.710 INFO:journalctl@ceph.mgr.x.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:47:22.854 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:47:22.854 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', '71e641de-0d84-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-05-08T21:47:22.893 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:47:22.894 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T21:47:22.894 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2024-05-08T21:47:22.928 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:47:22.928 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2024-05-08T21:47:22.928 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2024-05-08T21:47:22.935 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T21:47:22.935 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T21:47:22.941 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T21:47:22.941 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-08T21:47:22.947 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-05-08T21:47:22.953 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-05-08T21:47:22.954 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-05-08T21:47:22.954 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2024-05-08T21:47:22.989 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:47:22.989 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T21:47:22.989 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2024-05-08T21:47:22.989 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2024-05-08T21:47:22.995 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T21:47:22.995 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T21:47:23.001 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T21:47:23.001 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-08T21:47:23.007 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-05-08T21:47:23.013 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-05-08T21:47:23.013 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-05-08T21:47:23.013 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2024-05-08T21:47:23.013 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:23.014 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 139759432014312 on /run/cephadm/71e641de-0d84-11ef-bc97-c7b262605968.lock 2024-05-08T21:47:23.014 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 139759432014312 acquired on /run/cephadm/71e641de-0d84-11ef-bc97-c7b262605968.lock 2024-05-08T21:47:23.014 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2024-05-08T21:47:23.014 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2024-05-08T21:47:23.014 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-05-08T21:47:23.018 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-05-08T21:47:23.018 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-05-08T21:47:23.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T21:47:23.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-08T21:47:23.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-08T21:47:23.024 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T21:47:23.024 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T21:47:23.024 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T21:47:23.024 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-08T21:47:23.024 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::2cf5:5cd7:3838:489e/64 scope link noprefixroute 2024-05-08T21:47:23.024 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T21:47:23.025 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-05-08T21:47:23.025 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-05-08T21:47:23.025 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T21:47:23.025 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T21:47:23.026 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T21:47:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T21:47:24.101 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T21:47:24.101 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T21:47:24.101 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T21:47:24.755 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2024-05-08T21:47:24.755 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2024-05-08T21:47:25.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDs8jtmqgBNOxAAEn/smyb89GMXAYIXpNX1hw== 2024-05-08T21:47:26.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDt8jtmK4LtJRAA0JX8XilkHgv+UjQyQreAfw== 2024-05-08T21:47:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDu8jtmalKMERAAyDukoAilWVyrGMQ76TJHew== 2024-05-08T21:47:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2024-05-08T21:47:27.260 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = pacific 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:47:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2024-05-08T21:47:27.861 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2024-05-08T21:47:28.008 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T21:47:28.102 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T21:47:28.237 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-71e641de-0d84-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-71e641de-0d84-11ef-bc97-c7b262605968.target. 2024-05-08T21:47:28.237 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-71e641de-0d84-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-71e641de-0d84-11ef-bc97-c7b262605968.target. 2024-05-08T21:47:28.435 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a 2024-05-08T21:47:28.435 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T21:47:28.530 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-71e641de-0d84-11ef-bc97-c7b262605968.target.wants/ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-71e641de-0d84-11ef-bc97-c7b262605968@.service. 2024-05-08T21:47:28.572 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:28 smithi064 systemd[1]: Starting Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:47:28.930 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:47:28.931 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-08T21:47:28.939 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:47:28.939 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-08T21:47:28.939 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-08T21:47:28.939 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T21:47:28.939 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2024-05-08T21:47:28.939 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2024-05-08T21:47:29.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:28 smithi064 bash[41979]: f17e8356ef8919905e8ee4082801c69cfb101e20f97bd404ec4c7aa4f4ada302 2024-05-08T21:47:29.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:28 smithi064 systemd[1]: Started Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:47:29.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:28 smithi064 ceph-mon[42016]: mkfs 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:29.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:28 smithi064 ceph-mon[42016]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:47:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.429234s) 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T21:47:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.294+0000 7fe6d8249700 1 Processor -- start 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.294+0000 7fe6d8249700 1 -- start start 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.295+0000 7fe6d8249700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d0105ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.295+0000 7fe6d8249700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0106090 con 0x7fe6d01056a0 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.295+0000 7fe6d5fe5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d0105ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.295+0000 7fe6d5fe5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d0105ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44102/0 (socket says 172.21.15.64:44102) 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.295+0000 7fe6d5fe5700 1 -- 172.21.15.64:0/3219999650 learned_addr learned my addr 172.21.15.64:0/3219999650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.295+0000 7fe6d5fe5700 1 -- 172.21.15.64:0/3219999650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d01068a0 con 0x7fe6d01056a0 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d5fe5700 1 --2- 172.21.15.64:0/3219999650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d0105ac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0098a0 tx=0x7fe6cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa929449d04335e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d4fe3700 1 -- 172.21.15.64:0/3219999650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc004160 con 0x7fe6d01056a0 2024-05-08T21:47:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d4fe3700 1 -- 172.21.15.64:0/3219999650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc0042c0 con 0x7fe6d01056a0 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d4fe3700 1 -- 172.21.15.64:0/3219999650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc0045a0 con 0x7fe6d01056a0 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d8249700 1 -- 172.21.15.64:0/3219999650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 msgr2=0x7fe6d0105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d8249700 1 --2- 172.21.15.64:0/3219999650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d0105ac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0098a0 tx=0x7fe6cc009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d8249700 1 -- 172.21.15.64:0/3219999650 shutdown_connections 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d8249700 1 --2- 172.21.15.64:0/3219999650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d0105ac0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d8249700 1 -- 172.21.15.64:0/3219999650 >> 172.21.15.64:0/3219999650 conn(0x7fe6d0100c40 msgr2=0x7fe6d0103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.296+0000 7fe6d8249700 1 -- 172.21.15.64:0/3219999650 shutdown_connections 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.297+0000 7fe6d8249700 1 -- 172.21.15.64:0/3219999650 wait complete. 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.297+0000 7fe6d8249700 1 Processor -- start 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.297+0000 7fe6d8249700 1 -- start start 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.297+0000 7fe6d8249700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d018e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.297+0000 7fe6d8249700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d018e750 con 0x7fe6d01056a0 2024-05-08T21:47:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d5fe5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d018e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d5fe5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d018e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44108/0 (socket says 172.21.15.64:44108) 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d5fe5700 1 -- 172.21.15.64:0/1443247299 learned_addr learned my addr 172.21.15.64:0/1443247299 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d5fe5700 1 -- 172.21.15.64:0/1443247299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc009550 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d5fe5700 1 --2- 172.21.15.64:0/1443247299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d018e210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0106330 tx=0x7fe6cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc003da0 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc011630 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d018c990 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc011910 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d018ce30 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe6cc022070 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.298+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe6cc01b7b0 con 0x7fe6d01056a0 2024-05-08T21:47:29.691 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.299+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d00625d0 con 0x7fe6d01056a0 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.302+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe6cc025470 con 0x7fe6d01056a0 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.375+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe6d0106e20 con 0x7fe6d01056a0 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.375+0000 7fe6c6ffd700 1 -- 172.21.15.64:0/1443247299 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe6cc025610 con 0x7fe6d01056a0 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 msgr2=0x7fe6d018e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 --2- 172.21.15.64:0/1443247299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d018e210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0106330 tx=0x7fe6cc004750 comp rx=0 tx=0).stop 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 shutdown_connections 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 --2- 172.21.15.64:0/1443247299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d01056a0 0x7fe6d018e210 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 >> 172.21.15.64:0/1443247299 conn(0x7fe6d0100c40 msgr2=0x7fe6d0190ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 shutdown_connections 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:29.376+0000 7fe6d8249700 1 -- 172.21.15.64:0/1443247299 wait complete. 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2024-05-08T21:47:29.692 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T21:47:30.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:29 smithi064 ceph-mon[42016]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:47:30.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:29 smithi064 ceph-mon[42016]: monmap e1: 1 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:47:30.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:29 smithi064 ceph-mon[42016]: fsmap 2024-05-08T21:47:30.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:29 smithi064 ceph-mon[42016]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T21:47:30.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:29 smithi064 ceph-mon[42016]: mgrmap e1: no daemons active 2024-05-08T21:47:30.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:29 smithi064 ceph-mon[42016]: from='client.? 172.21.15.64:0/1443247299' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.101+0000 7f1f8c9be700 1 Processor -- start 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.101+0000 7f1f8c9be700 1 -- start start 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.101+0000 7f1f8c9be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88109720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.101+0000 7f1f8c9be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88079df0 con 0x7f1f88107330 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.101+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88109720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88109720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44120/0 (socket says 172.21.15.64:44120) 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8659c700 1 -- 172.21.15.64:0/2116937052 learned_addr learned my addr 172.21.15.64:0/2116937052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8659c700 1 -- 172.21.15.64:0/2116937052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88079f30 con 0x7f1f88107330 2024-05-08T21:47:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8659c700 1 --2- 172.21.15.64:0/2116937052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88109720 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0098a0 tx=0x7f1f7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab61979e7aceac4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8559a700 1 -- 172.21.15.64:0/2116937052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f7c004160 con 0x7f1f88107330 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8559a700 1 -- 172.21.15.64:0/2116937052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0042c0 con 0x7f1f88107330 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.102+0000 7f1f8559a700 1 -- 172.21.15.64:0/2116937052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f7c0045a0 con 0x7f1f88107330 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2116937052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 msgr2=0x7f1f88109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 --2- 172.21.15.64:0/2116937052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88109720 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0098a0 tx=0x7f1f7c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2116937052 shutdown_connections 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 --2- 172.21.15.64:0/2116937052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88109720 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2116937052 >> 172.21.15.64:0/2116937052 conn(0x7f1f88100c40 msgr2=0x7f1f881030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2116937052 shutdown_connections 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2116937052 wait complete. 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.103+0000 7f1f8c9be700 1 Processor -- start 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8c9be700 1 -- start start 2024-05-08T21:47:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8c9be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8c9be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819a230 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44124/0 (socket says 172.21.15.64:44124) 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8659c700 1 -- 172.21.15.64:0/2381692534 learned_addr learned my addr 172.21.15.64:0/2381692534 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8659c700 1 -- 172.21.15.64:0/2381692534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c009550 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8659c700 1 --2- 172.21.15.64:0/2381692534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88199cf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1f88079230 tx=0x7f1f7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f7c011630 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c011790 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.104+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8819a430 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.105+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f7c011a30 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.105+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8819a8d0 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.105+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1f7c021070 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.105+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f7c01b780 con 0x7f1f88107330 2024-05-08T21:47:30.553 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.106+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f88193ae0 con 0x7f1f88107330 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.108+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f1f7c01acc0 con 0x7f1f88107330 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.178+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1f88109e40 con 0x7f1f88107330 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.182+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f7c01ae60 con 0x7f1f88107330 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.182+0000 7f1f777fe700 1 -- 172.21.15.64:0/2381692534 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f1f7c0334e0 con 0x7f1f88107330 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 msgr2=0x7f1f88199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 --2- 172.21.15.64:0/2381692534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88199cf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1f88079230 tx=0x7f1f7c004750 comp rx=0 tx=0).stop 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 shutdown_connections 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 --2- 172.21.15.64:0/2381692534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f88107330 0x7f1f88199cf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 >> 172.21.15.64:0/2381692534 conn(0x7f1f88100c40 msgr2=0x7f1f88102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 shutdown_connections 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.183+0000 7f1f8c9be700 1 -- 172.21.15.64:0/2381692534 wait complete. 2024-05-08T21:47:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2024-05-08T21:47:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:30 smithi064 ceph-mon[42016]: from='client.? 172.21.15.64:0/2381692534' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T21:47:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:30 smithi064 ceph-mon[42016]: from='client.? 172.21.15.64:0/2381692534' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T21:47:31.339 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.884+0000 7f617ba1b700 1 Processor -- start 2024-05-08T21:47:31.339 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.884+0000 7f617ba1b700 1 -- start start 2024-05-08T21:47:31.339 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f617ba1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f6174105ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:31.339 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f617ba1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6174106090 con 0x7f61741056a0 2024-05-08T21:47:31.339 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f61797b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f6174105ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:31.339 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f61797b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f6174105ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44138/0 (socket says 172.21.15.64:44138) 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f61797b7700 1 -- 172.21.15.64:0/1581906401 learned_addr learned my addr 172.21.15.64:0/1581906401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f61797b7700 1 -- 172.21.15.64:0/1581906401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61741068a0 con 0x7f61741056a0 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f61797b7700 1 --2- 172.21.15.64:0/1581906401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f6174105ac0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f616400bb40 tx=0x7f616400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d5baff707b3eed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f616bfff700 1 -- 172.21.15.64:0/1581906401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f616400dbe0 con 0x7f61741056a0 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f616bfff700 1 -- 172.21.15.64:0/1581906401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61640044d0 con 0x7f61741056a0 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.885+0000 7f616bfff700 1 -- 172.21.15.64:0/1581906401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6164010470 con 0x7f61741056a0 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 -- 172.21.15.64:0/1581906401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 msgr2=0x7f6174105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 --2- 172.21.15.64:0/1581906401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f6174105ac0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f616400bb40 tx=0x7f616400be50 comp rx=0 tx=0).stop 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 -- 172.21.15.64:0/1581906401 shutdown_connections 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 --2- 172.21.15.64:0/1581906401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f6174105ac0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 -- 172.21.15.64:0/1581906401 >> 172.21.15.64:0/1581906401 conn(0x7f6174100c40 msgr2=0x7f6174103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 -- 172.21.15.64:0/1581906401 shutdown_connections 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.886+0000 7f617ba1b700 1 -- 172.21.15.64:0/1581906401 wait complete. 2024-05-08T21:47:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f617ba1b700 1 Processor -- start 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f617ba1b700 1 -- start start 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f617ba1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f61741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f617ba1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6174198900 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f61797b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f61741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f61797b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f61741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44148/0 (socket says 172.21.15.64:44148) 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f61797b7700 1 -- 172.21.15.64:0/497176665 learned_addr learned my addr 172.21.15.64:0/497176665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f61797b7700 1 -- 172.21.15.64:0/497176665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616400b7f0 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.887+0000 7f61797b7700 1 --2- 172.21.15.64:0/497176665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f61741983c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6164003c60 tx=0x7f6164003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f616401e070 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6174198b00 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6164004250 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6164022900 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6174198fa0 con 0x7f61741056a0 2024-05-08T21:47:31.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f616402c440 con 0x7f61741056a0 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.888+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6164019580 con 0x7f61741056a0 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.889+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61740625d0 con 0x7f61741056a0 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.891+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f6164027070 con 0x7f61741056a0 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.962+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6174073970 con 0x7f61741056a0 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.962+0000 7f616a7fc700 1 -- 172.21.15.64:0/497176665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f616402c8d0 con 0x7f61741056a0 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 msgr2=0x7f61741983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 --2- 172.21.15.64:0/497176665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f61741983c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6164003c60 tx=0x7f6164003980 comp rx=0 tx=0).stop 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 shutdown_connections 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 --2- 172.21.15.64:0/497176665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61741056a0 0x7f61741983c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 >> 172.21.15.64:0/497176665 conn(0x7f6174100c40 msgr2=0x7f617418ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 shutdown_connections 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:30.963+0000 7f617ba1b700 1 -- 172.21.15.64:0/497176665 wait complete. 2024-05-08T21:47:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2024-05-08T21:47:31.546 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:31 smithi064 systemd[1]: Stopping Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:47:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:31 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a[41993]: 2024-05-08T21:47:31.545+0000 7f3a07223700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:47:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:31 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a[41993]: 2024-05-08T21:47:31.545+0000 7f3a07223700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T21:47:32.393 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 bash[42342]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a 2024-05-08T21:47:32.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 systemd[1]: ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T21:47:32.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 systemd[1]: Stopped Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:47:32.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 systemd[1]: Starting Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:47:32.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T21:47:32.728 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: pidfile_write: ignore empty --pid-file 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 bash[42462]: 38c1f9ac2a9ddfc509de7d982d7174fbbde8b9fffc82582b5b59d1dfb3a461ec 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: load: jerasure load: lrc 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: RocksDB version: 7.9.2 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Git sha 0 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: DB SUMMARY 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: DB Session ID: H551MWE8U2XUF04HYD2X 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: CURRENT file: CURRENT 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T21:47:33.204 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98385 ; 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.error_if_exists: 0 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.create_if_missing: 0 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.paranoid_checks: 1 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.env: 0x55e6d90ea820 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.info_log: 0x55e6da8bb520 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T21:47:33.205 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.statistics: (nil) 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.use_fsync: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_log_file_size: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.allow_fallocate: 1 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.use_direct_reads: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.db_log_dir: 2024-05-08T21:47:33.206 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.wal_dir: 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.write_buffer_manager: 0x55e6d9b50320 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T21:47:33.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.unordered_write: 0 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.row_cache: None 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.wal_filter: None 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T21:47:33.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.two_write_queues: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.wal_compression: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.atomic_flush: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.log_readahead_size: 0 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T21:47:33.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T21:47:33.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T21:47:33.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T21:47:33.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T21:47:33.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_background_jobs: 2 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_background_compactions: -1 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_subcompactions: 1 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T21:47:33.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_open_files: -1 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_background_flushes: -1 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Compression algorithms supported: 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T21:47:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kZSTD supported: 0 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kXpressCompression supported: 0 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kZlibCompression supported: 1 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kSnappyCompression supported: 1 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kLZ4Compression supported: 1 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: kBZip2Compression supported: 0 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.merge_operator: 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_filter: None 2024-05-08T21:47:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e6da8bb620) 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_type: 0 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_index_type: 0 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_shortening: 1 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: checksum: 4 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: no_block_cache: 0 2024-05-08T21:47:33.214 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache: 0x55e6d9bd31f0 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_options: 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: capacity : 536870912 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_shard_bits : 4 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: strict_capacity_limit : 0 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_compressed: (nil) 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: persistent_cache: (nil) 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size: 4096 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size_deviation: 10 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_restart_interval: 16 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_block_restart_interval: 1 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: metadata_block_size: 4096 2024-05-08T21:47:33.215 INFO:journalctl@ceph.mon.a.smithi064.stdout: partition_filters: 0 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: use_delta_encoding: 1 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: filter_policy: bloomfilter 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: whole_key_filtering: 1 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: verify_compression: 0 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: format_version: 5 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_index_compression: 1 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_align: 0 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: max_auto_readahead_size: 262144 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: prepopulate_block_cache: 0 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: initial_auto_readahead_size: 8192 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression: NoCompression 2024-05-08T21:47:33.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.num_levels: 7 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:47:33.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T21:47:33.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T21:47:33.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.table_properties_collectors: 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.inplace_update_support: 0 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T21:47:33.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.bloom_locality: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.max_successive_merges: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.ttl: 2592000 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.enable_blob_files: false 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.min_blob_size: 0 2024-05-08T21:47:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 systemd[1]: Started Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7aaaf4ce-5f79-4a2e-86d8-8106fbf57d23 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715204852726925, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T21:47:33.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715204852728416, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93701, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 311, "table_properties": {"data_size": 91692, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13824, "raw_average_key_size": 50, "raw_value_size": 84172, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715204852, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7aaaf4ce-5f79-4a2e-86d8-8106fbf57d23", "db_session_id": "H551MWE8U2XUF04HYD2X", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715204852728488, "job": 1, "event": "recovery_finished"} 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e6d9c70000 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: DB pointer 0x55e6d9c5e000 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** DB Stats ** 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:47:33.223 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: L0 2/0 93.34 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: Sum 2/0 93.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 107.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-08T21:47:33.224 INFO:journalctl@ceph.mon.a.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative compaction: 0.00 GB write, 22.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval compaction: 0.00 GB write, 22.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache BinnedLRUCache@0x55e6d9bd31f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T21:47:33.225 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???) e1 preinit fsid 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).mds e1 new map 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).mds e1 print_map 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout: e1 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout: legacy client fscid: -1 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout: No filesystems configured 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T21:47:33.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:47:33.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: monmap e1: 1 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:47:33.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: fsmap 2024-05-08T21:47:33.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T21:47:33.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:32 smithi064 ceph-mon[42500]: mgrmap e1: no daemons active 2024-05-08T21:47:33.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.097+0000 7fa89c2c9700 1 Processor -- start 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.097+0000 7fa89c2c9700 1 -- start start 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.098+0000 7fa89c2c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa8941054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.098+0000 7fa89c2c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8940747d0 con 0x7fa894105090 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.098+0000 7fa89a065700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa8941054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.098+0000 7fa89a065700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa8941054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49424/0 (socket says 172.21.15.64:49424) 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.098+0000 7fa89a065700 1 -- 172.21.15.64:0/629242074 learned_addr learned my addr 172.21.15.64:0/629242074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.098+0000 7fa89a065700 1 -- 172.21.15.64:0/629242074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8941059f0 con 0x7fa894105090 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89a065700 1 --2- 172.21.15.64:0/629242074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa8941054b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa8900098a0 tx=0x7fa890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4311cf4781babe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa899063700 1 -- 172.21.15.64:0/629242074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa890004160 con 0x7fa894105090 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa899063700 1 -- 172.21.15.64:0/629242074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa8900042c0 con 0x7fa894105090 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa899063700 1 -- 172.21.15.64:0/629242074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa890011a00 con 0x7fa894105090 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89c2c9700 1 -- 172.21.15.64:0/629242074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 msgr2=0x7fa8941054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:33.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89c2c9700 1 --2- 172.21.15.64:0/629242074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa8941054b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa8900098a0 tx=0x7fa890009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89c2c9700 1 -- 172.21.15.64:0/629242074 shutdown_connections 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89c2c9700 1 --2- 172.21.15.64:0/629242074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa8941054b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89c2c9700 1 -- 172.21.15.64:0/629242074 >> 172.21.15.64:0/629242074 conn(0x7fa894100c50 msgr2=0x7fa894103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.099+0000 7fa89c2c9700 1 -- 172.21.15.64:0/629242074 shutdown_connections 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89c2c9700 1 -- 172.21.15.64:0/629242074 wait complete. 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89c2c9700 1 Processor -- start 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89c2c9700 1 -- start start 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89c2c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa89419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89c2c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419cc80 con 0x7fa894105090 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89a065700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa89419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89a065700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa89419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49428/0 (socket says 172.21.15.64:49428) 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.100+0000 7fa89a065700 1 -- 172.21.15.64:0/2712256909 learned_addr learned my addr 172.21.15.64:0/2712256909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:33.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa89a065700 1 -- 172.21.15.64:0/2712256909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009550 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa89a065700 1 --2- 172.21.15.64:0/2712256909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa89419c740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa890009fb0 tx=0x7fa890011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa890003a40 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa890003ba0 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89419ce80 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89419d320 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa89001a750 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa89001ac80 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.101+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa89002de40 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.102+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878005320 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.104+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa890024b60 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.175+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa8780058a0 con 0x7fa894105090 2024-05-08T21:47:33.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.180+0000 7fa8877fe700 1 -- 172.21.15.64:0/2712256909 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa890024d00 con 0x7fa894105090 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 msgr2=0x7fa89419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 --2- 172.21.15.64:0/2712256909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa89419c740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa890009fb0 tx=0x7fa890011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 shutdown_connections 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 --2- 172.21.15.64:0/2712256909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa894105090 0x7fa89419c740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 >> 172.21.15.64:0/2712256909 conn(0x7fa894100c50 msgr2=0x7fa8941935f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 shutdown_connections 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:33.181+0000 7fa89c2c9700 1 -- 172.21.15.64:0/2712256909 wait complete. 2024-05-08T21:47:33.551 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T21:47:33.577 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:47:33.577 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2024-05-08T21:47:33.577 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T21:47:33.577 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T21:47:33.774 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x 2024-05-08T21:47:33.774 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T21:47:33.874 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-71e641de-0d84-11ef-bc97-c7b262605968.target.wants/ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-71e641de-0d84-11ef-bc97-c7b262605968@.service. 2024-05-08T21:47:34.130 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:33 smithi064 systemd[1]: Starting Ceph mgr.x for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:47:34.289 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:47:34.289 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-08T21:47:34.297 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:47:34.297 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-08T21:47:34.297 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-08T21:47:34.297 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T21:47:34.306 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:47:34.306 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-08T21:47:34.314 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:47:34.314 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-08T21:47:34.315 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-08T21:47:34.315 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T21:47:34.315 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2024-05-08T21:47:34.315 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2024-05-08T21:47:34.437 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:34 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2712256909' entity='client.admin' 2024-05-08T21:47:34.437 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:34 smithi064 bash[42737]: 31d83bc8920ab52e1a995fac846e8f1eb0d500af04d31edd275bd2f7792e1167 2024-05-08T21:47:34.437 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:34 smithi064 systemd[1]: Started Ceph mgr.x for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:47:34.973 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:34.710+0000 7fd411317200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:47:34.973 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:34.809+0000 7fd411317200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:47:34.974 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:34.903+0000 7fd411317200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:47:35.144 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:35.144 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:47:35.145 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:47:35.145 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:47:35.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:47:35.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:47:35.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:35.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:47:28.949561+0000", 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.706+0000 7fdd5de61700 1 Processor -- start 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.706+0000 7fdd5de61700 1 -- start start 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.707+0000 7fdd5de61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd581096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.707+0000 7fdd5de61700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58079f30 con 0x7fdd581072f0 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.707+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd581096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.707+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd581096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49458/0 (socket says 172.21.15.64:49458) 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.707+0000 7fdd577fe700 1 -- 172.21.15.64:0/119574871 learned_addr learned my addr 172.21.15.64:0/119574871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.707+0000 7fdd577fe700 1 -- 172.21.15.64:0/119574871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58109c20 con 0x7fdd581072f0 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.708+0000 7fdd577fe700 1 --2- 172.21.15.64:0/119574871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd581096e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098a0 tx=0x7fdd48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6076749b7feaf2a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.708+0000 7fdd567fc700 1 -- 172.21.15.64:0/119574871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd48004160 con 0x7fdd581072f0 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.708+0000 7fdd567fc700 1 -- 172.21.15.64:0/119574871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd480042c0 con 0x7fdd581072f0 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.708+0000 7fdd567fc700 1 -- 172.21.15.64:0/119574871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd48011a00 con 0x7fdd581072f0 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 -- 172.21.15.64:0/119574871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 msgr2=0x7fdd581096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 --2- 172.21.15.64:0/119574871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd581096e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098a0 tx=0x7fdd48009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 -- 172.21.15.64:0/119574871 shutdown_connections 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 --2- 172.21.15.64:0/119574871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd581096e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 -- 172.21.15.64:0/119574871 >> 172.21.15.64:0/119574871 conn(0x7fdd58100cc0 msgr2=0x7fdd581030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 -- 172.21.15.64:0/119574871 shutdown_connections 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 -- 172.21.15.64:0/119574871 wait complete. 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.709+0000 7fdd5de61700 1 Processor -- start 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd5de61700 1 -- start start 2024-05-08T21:47:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd5de61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd58106b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd5de61700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58107060 con 0x7fdd581072f0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd58106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd58106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49470/0 (socket says 172.21.15.64:49470) 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd577fe700 1 -- 172.21.15.64:0/3043357769 learned_addr learned my addr 172.21.15.64:0/3043357769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd577fe700 1 -- 172.21.15.64:0/3043357769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009550 con 0x7fdd581072f0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.710+0000 7fdd577fe700 1 --2- 172.21.15.64:0/3043357769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd58106b20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009130 tx=0x7fdd48011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd48003b70 con 0x7fdd581072f0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd48003cd0 con 0x7fdd581072f0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd581052a0 con 0x7fdd581072f0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd480197d0 con 0x7fdd581072f0 2024-05-08T21:47:35.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd58105740 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdd480199c0 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.711+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd4800b720 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.712+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd580625d0 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.714+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdd4802a060 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.789+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdd5810aa80 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.789+0000 7fdd54ff9700 1 -- 172.21.15.64:0/3043357769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fdd48019e00 con 0x7fdd581072f0 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.790+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 msgr2=0x7fdd58106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.790+0000 7fdd5de61700 1 --2- 172.21.15.64:0/3043357769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd58106b20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009130 tx=0x7fdd48011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.790+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 shutdown_connections 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.790+0000 7fdd5de61700 1 --2- 172.21.15.64:0/3043357769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd581072f0 0x7fdd58106b20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.790+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 >> 172.21.15.64:0/3043357769 conn(0x7fdd58100cc0 msgr2=0x7fdd58101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.791+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 shutdown_connections 2024-05-08T21:47:35.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:34.791+0000 7fdd5de61700 1 -- 172.21.15.64:0/3043357769 wait complete. 2024-05-08T21:47:35.165 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2024-05-08T21:47:35.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:35 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3043357769' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:47:35.463 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:34.997+0000 7fd411317200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:47:35.945 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:35.682+0000 7fd411317200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:47:36.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:35.944+0000 7fd411317200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:47:36.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:36.034+0000 7fd411317200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:47:36.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:36.149+0000 7fd411317200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:47:36.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:36.312+0000 7fd411317200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:47:36.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:36.445+0000 7fd411317200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:47:36.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:36.534+0000 7fd411317200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:47:37.212 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:36.922+0000 7fd411317200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:47:37.643 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:37.336+0000 7fd411317200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:47:37.643 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:37.544+0000 7fd411317200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:47:37.643 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:37.642+0000 7fd411317200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:47:38.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:38 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/512666290' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:47:38.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:38.466+0000 7fd411317200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:47:38.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:38.567+0000 7fd411317200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:47:39.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:38.723+0000 7fd411317200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:47:39.535 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:39.535 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:47:39.535 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:47:39.535 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:47:39.537 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:47:39.537 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.537 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:47:39.537 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:39.537 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:47:39.537 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:47:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:47:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:47:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:47:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:47:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:47:39.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:47:28.949561+0000", 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf65bc9700 1 Processor -- start 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf65bc9700 1 -- start start 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf65bc9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf60107150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf65bc9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf60107720 con 0x7fbf60106d30 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf5f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf60107150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf5f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf60107150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49486/0 (socket says 172.21.15.64:49486) 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.245+0000 7fbf5f7fe700 1 -- 172.21.15.64:0/2301338058 learned_addr learned my addr 172.21.15.64:0/2301338058 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.246+0000 7fbf5f7fe700 1 -- 172.21.15.64:0/2301338058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60107860 con 0x7fbf60106d30 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.246+0000 7fbf5f7fe700 1 --2- 172.21.15.64:0/2301338058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf60107150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf500098a0 tx=0x7fbf50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efe8a9cb684d704 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.246+0000 7fbf5e7fc700 1 -- 172.21.15.64:0/2301338058 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf50004160 con 0x7fbf60106d30 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.246+0000 7fbf5e7fc700 1 -- 172.21.15.64:0/2301338058 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf500042c0 con 0x7fbf60106d30 2024-05-08T21:47:39.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.246+0000 7fbf5e7fc700 1 -- 172.21.15.64:0/2301338058 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf50011a00 con 0x7fbf60106d30 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 -- 172.21.15.64:0/2301338058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 msgr2=0x7fbf60107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 --2- 172.21.15.64:0/2301338058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf60107150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf500098a0 tx=0x7fbf50009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 -- 172.21.15.64:0/2301338058 shutdown_connections 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 --2- 172.21.15.64:0/2301338058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf60107150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 -- 172.21.15.64:0/2301338058 >> 172.21.15.64:0/2301338058 conn(0x7fbf60100390 msgr2=0x7fbf60102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 -- 172.21.15.64:0/2301338058 shutdown_connections 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 -- 172.21.15.64:0/2301338058 wait complete. 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 Processor -- start 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.247+0000 7fbf65bc9700 1 -- start start 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf65bc9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf6019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf65bc9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6019cc20 con 0x7fbf60106d30 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf6019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf6019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49502/0 (socket says 172.21.15.64:49502) 2024-05-08T21:47:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5f7fe700 1 -- 172.21.15.64:0/512666290 learned_addr learned my addr 172.21.15.64:0/512666290 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5f7fe700 1 -- 172.21.15.64:0/512666290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50009550 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5f7fe700 1 --2- 172.21.15.64:0/512666290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf6019c6e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009fb0 tx=0x7fbf50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf50003a40 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.248+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf50003ba0 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.249+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6019ce20 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.249+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf5001a750 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.249+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6019d2c0 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.249+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fbf5001ac80 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.249+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbf500193c0 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.250+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf601054a0 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.252+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fbf50024b60 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.345+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbf60105660 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.345+0000 7fbf5cff9700 1 -- 172.21.15.64:0/512666290 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbf50024d00 con 0x7fbf60106d30 2024-05-08T21:47:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.346+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 msgr2=0x7fbf6019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.346+0000 7fbf65bc9700 1 --2- 172.21.15.64:0/512666290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf6019c6e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009fb0 tx=0x7fbf50011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.346+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 shutdown_connections 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.346+0000 7fbf65bc9700 1 --2- 172.21.15.64:0/512666290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf60106d30 0x7fbf6019c6e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.346+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 >> 172.21.15.64:0/512666290 conn(0x7fbf60100390 msgr2=0x7fbf60192f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.347+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 shutdown_connections 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:38.347+0000 7fbf65bc9700 1 -- 172.21.15.64:0/512666290 wait complete. 2024-05-08T21:47:39.551 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2024-05-08T21:47:39.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:39.385+0000 7fd411317200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:47:39.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:39.578+0000 7fd411317200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:47:40.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:39.781+0000 7fd411317200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:47:41.059 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:40 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:40.760+0000 7fd411317200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:47:41.059 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:41.058+0000 7fd411317200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:47:41.354 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:41.151+0000 7fd411317200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:47:41.354 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:41.353+0000 7fd411317200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:47:41.639 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: Activating manager daemon x 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: mgrmap e2: x(active, starting, since 0.00324266s) 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: Manager daemon x is now available 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' 2024-05-08T21:47:41.640 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:41 smithi064 ceph-mon[42500]: from='mgr.14100 172.21.15.64:0/2894453775' entity='mgr.x' 2024-05-08T21:47:42.277 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:42.277 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:47:42.278 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:47:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:47:42.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:47:42.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:47:42.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:47:42.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:47:42.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:47:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:42.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:47:28.949561+0000", 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.857+0000 7fd8bae3f700 1 Processor -- start 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.857+0000 7fd8bae3f700 1 -- start start 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.857+0000 7fd8bae3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b4104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.857+0000 7fd8bae3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4105490 con 0x7fd8b4104aa0 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b8bdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b4104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b8bdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b4104ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49578/0 (socket says 172.21.15.64:49578) 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b8bdb700 1 -- 172.21.15.64:0/3666661536 learned_addr learned my addr 172.21.15.64:0/3666661536 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b8bdb700 1 -- 172.21.15.64:0/3666661536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4107a30 con 0x7fd8b4104aa0 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b8bdb700 1 --2- 172.21.15.64:0/3666661536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b4104ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098a0 tx=0x7fd8a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a135cb0aa075c316 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b37fe700 1 -- 172.21.15.64:0/3666661536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a8004160 con 0x7fd8b4104aa0 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b37fe700 1 -- 172.21.15.64:0/3666661536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8a80042c0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.858+0000 7fd8b37fe700 1 -- 172.21.15.64:0/3666661536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a8011a00 con 0x7fd8b4104aa0 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 -- 172.21.15.64:0/3666661536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 msgr2=0x7fd8b4104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 --2- 172.21.15.64:0/3666661536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b4104ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098a0 tx=0x7fd8a8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 -- 172.21.15.64:0/3666661536 shutdown_connections 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 --2- 172.21.15.64:0/3666661536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b4104ec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 -- 172.21.15.64:0/3666661536 >> 172.21.15.64:0/3666661536 conn(0x7fd8b41003f0 msgr2=0x7fd8b4102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 -- 172.21.15.64:0/3666661536 shutdown_connections 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 -- 172.21.15.64:0/3666661536 wait complete. 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.859+0000 7fd8bae3f700 1 Processor -- start 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8bae3f700 1 -- start start 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8bae3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8bae3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b419ccf0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b8bdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b8bdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49580/0 (socket says 172.21.15.64:49580) 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b8bdb700 1 -- 172.21.15.64:0/477066843 learned_addr learned my addr 172.21.15.64:0/477066843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b8bdb700 1 -- 172.21.15.64:0/477066843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8009550 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b8bdb700 1 --2- 172.21.15.64:0/477066843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b419c7b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009130 tx=0x7fd8a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a8003ab0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8a8003c10 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b419cef0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.860+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a80247d0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.861+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b419d390 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.861+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 2) v1 ==== 45297+0+0 (secure 0 0 0) 0x7fd8a802d3f0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.862+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b410ac60 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.862+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd8a804bfa0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.864+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd8a802d6a0 con 0x7fd8b4104aa0 2024-05-08T21:47:42.292 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.937+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd8b419fc80 con 0x7fd8b4104aa0 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.938+0000 7fd8b1ffb700 1 -- 172.21.15.64:0/477066843 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd8a8010380 con 0x7fd8b4104aa0 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.938+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 msgr2=0x7fd8b419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.938+0000 7fd8bae3f700 1 --2- 172.21.15.64:0/477066843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b419c7b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009130 tx=0x7fd8a8011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.938+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 shutdown_connections 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.938+0000 7fd8bae3f700 1 --2- 172.21.15.64:0/477066843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b4104aa0 0x7fd8b419c7b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.938+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 >> 172.21.15.64:0/477066843 conn(0x7fd8b41003f0 msgr2=0x7fd8b4100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.939+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 shutdown_connections 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:41.939+0000 7fd8bae3f700 1 -- 172.21.15.64:0/477066843 wait complete. 2024-05-08T21:47:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (3/15)... 2024-05-08T21:47:42.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:42 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/477066843' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:47:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:42 smithi064 ceph-mon[42500]: mgrmap e3: x(active, since 1.00621s) 2024-05-08T21:47:44.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:44 smithi064 ceph-mon[42500]: mgrmap e4: x(active, since 2s) 2024-05-08T21:47:45.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:45 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3885699488' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:47:48.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:48.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:47:48.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:48.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:47:48.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:47:48.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:48.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:47:28.949561+0000", 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985fae1700 1 Processor -- start 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985fae1700 1 -- start start 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985fae1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f9858105ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985fae1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858106090 con 0x7f98581056a0 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985d87d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f9858105ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985d87d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f9858105ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52508/0 (socket says 172.21.15.64:52508) 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.058+0000 7f985d87d700 1 -- 172.21.15.64:0/3478230728 learned_addr learned my addr 172.21.15.64:0/3478230728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985d87d700 1 -- 172.21.15.64:0/3478230728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98581068a0 con 0x7f98581056a0 2024-05-08T21:47:48.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985d87d700 1 --2- 172.21.15.64:0/3478230728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f9858105ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98540098a0 tx=0x7f9854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c414a0c0bc3c52c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985c87b700 1 -- 172.21.15.64:0/3478230728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9854004160 con 0x7f98581056a0 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985c87b700 1 -- 172.21.15.64:0/3478230728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98540042c0 con 0x7f98581056a0 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985c87b700 1 -- 172.21.15.64:0/3478230728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9854011a00 con 0x7f98581056a0 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985fae1700 1 -- 172.21.15.64:0/3478230728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 msgr2=0x7f9858105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985fae1700 1 --2- 172.21.15.64:0/3478230728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f9858105ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98540098a0 tx=0x7f9854009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985fae1700 1 -- 172.21.15.64:0/3478230728 shutdown_connections 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985fae1700 1 --2- 172.21.15.64:0/3478230728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f9858105ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.059+0000 7f985fae1700 1 -- 172.21.15.64:0/3478230728 >> 172.21.15.64:0/3478230728 conn(0x7f9858100c40 msgr2=0x7f9858103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985fae1700 1 -- 172.21.15.64:0/3478230728 shutdown_connections 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985fae1700 1 -- 172.21.15.64:0/3478230728 wait complete. 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985fae1700 1 Processor -- start 2024-05-08T21:47:48.024 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985fae1700 1 -- start start 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985fae1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f985819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985fae1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985819ab70 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985d87d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f985819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985d87d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f985819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52514/0 (socket says 172.21.15.64:52514) 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.060+0000 7f985d87d700 1 -- 172.21.15.64:0/3885699488 learned_addr learned my addr 172.21.15.64:0/3885699488 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f985d87d700 1 -- 172.21.15.64:0/3885699488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854009550 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f985d87d700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f985819a630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9854009130 tx=0x7f9854011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98540038b0 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9854003a10 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985819ad70 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9854019550 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.061+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985819b210 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.062+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f9854019810 con 0x7f98581056a0 2024-05-08T21:47:48.025 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.062+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9858194680 con 0x7f98581056a0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.062+0000 7f984effd700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f9844038640 0x7f984403ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.062+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f985404c3f0 con 0x7f98581056a0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.062+0000 7f985d07c700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f9844038640 0x7f984403ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.063+0000 7f985d07c700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f9844038640 0x7f984403ab00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9848006fd0 tx=0x7f9848006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.068+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9854019ac0 con 0x7f98581056a0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.360+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9858109ad0 con 0x7f98581056a0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.361+0000 7f984effd700 1 -- 172.21.15.64:0/3885699488 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9854018b40 con 0x7f98581056a0 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f9844038640 msgr2=0x7f984403ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f9844038640 0x7f984403ab00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9848006fd0 tx=0x7f9848006e40 comp rx=0 tx=0).stop 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 msgr2=0x7f985819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f985819a630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9854009130 tx=0x7f9854011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 shutdown_connections 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f9844038640 0x7f984403ab00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 --2- 172.21.15.64:0/3885699488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98581056a0 0x7f985819a630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:48.027 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 >> 172.21.15.64:0/3885699488 conn(0x7f9858100c40 msgr2=0x7f9858101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:48.027 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 shutdown_connections 2024-05-08T21:47:48.027 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:45.363+0000 7f985fae1700 1 -- 172.21.15.64:0/3885699488 wait complete. 2024-05-08T21:47:48.027 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2024-05-08T21:47:49.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:49 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2600782002' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T21:47:51.497 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:51.497 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T21:47:51.497 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:47:51.497 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T21:47:51.497 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.260+0000 7f63728fa700 1 Processor -- start 2024-05-08T21:47:51.498 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.260+0000 7f63728fa700 1 -- start start 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.260+0000 7f63728fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.260+0000 7f63728fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c1071f0 con 0x7f636c106800 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.260+0000 7f636bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52516/0 (socket says 172.21.15.64:52516) 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636bfff700 1 -- 172.21.15.64:0/3474689330 learned_addr learned my addr 172.21.15.64:0/3474689330 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636bfff700 1 -- 172.21.15.64:0/3474689330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c107a00 con 0x7f636c106800 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636bfff700 1 --2- 172.21.15.64:0/3474689330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c106c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098a0 tx=0x7f635c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d23c6e97b74d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636affd700 1 -- 172.21.15.64:0/3474689330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c004160 con 0x7f636c106800 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636affd700 1 -- 172.21.15.64:0/3474689330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f635c0042c0 con 0x7f636c106800 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.261+0000 7f636affd700 1 -- 172.21.15.64:0/3474689330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c011a00 con 0x7f636c106800 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 -- 172.21.15.64:0/3474689330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 msgr2=0x7f636c106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 --2- 172.21.15.64:0/3474689330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c106c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098a0 tx=0x7f635c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 -- 172.21.15.64:0/3474689330 shutdown_connections 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 --2- 172.21.15.64:0/3474689330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c106c20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:51.499 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 -- 172.21.15.64:0/3474689330 >> 172.21.15.64:0/3474689330 conn(0x7f636c101da0 msgr2=0x7f636c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 -- 172.21.15.64:0/3474689330 shutdown_connections 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 -- 172.21.15.64:0/3474689330 wait complete. 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 Processor -- start 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.262+0000 7f63728fa700 1 -- start start 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f63728fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f63728fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19ab40 con 0x7f636c106800 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f636bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f636bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52520/0 (socket says 172.21.15.64:52520) 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f636bfff700 1 -- 172.21.15.64:0/2600782002 learned_addr learned my addr 172.21.15.64:0/2600782002 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f636bfff700 1 -- 172.21.15.64:0/2600782002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c009550 con 0x7f636c106800 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f636bfff700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c19a600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f635c009130 tx=0x7f635c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c0038b0 con 0x7f636c106800 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c19ad40 con 0x7f636c106800 2024-05-08T21:47:51.500 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f635c003a10 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.263+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c019550 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.264+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c19b1e0 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.264+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f635c019810 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.264+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c10ac30 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.265+0000 7f63697fa700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f6358038640 0x7f635803ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.265+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f635c04c550 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.265+0000 7f636b7fe700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f6358038640 0x7f635803ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.265+0000 7f636b7fe700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f6358038640 0x7f635803ab00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6354006fd0 tx=0x7f6354006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.270+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f635c019ac0 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.469+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f636c19da70 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.470+0000 7f63697fa700 1 -- 172.21.15.64:0/2600782002 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f635c018350 con 0x7f636c106800 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f6358038640 msgr2=0x7f635803ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f6358038640 0x7f635803ab00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6354006fd0 tx=0x7f6354006e40 comp rx=0 tx=0).stop 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 msgr2=0x7f636c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c19a600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f635c009130 tx=0x7f635c011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 shutdown_connections 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f6358038640 0x7f635803ab00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 --2- 172.21.15.64:0/2600782002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c106800 0x7f636c19a600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 >> 172.21.15.64:0/2600782002 conn(0x7f636c101da0 msgr2=0x7f636c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 shutdown_connections 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:49.472+0000 7f63728fa700 1 -- 172.21.15.64:0/2600782002 wait complete. 2024-05-08T21:47:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2024-05-08T21:47:52.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:52 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/4189523467' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T21:47:52.963 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:52 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setuser ceph since I am not root 2024-05-08T21:47:52.963 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:52 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setgroup ceph since I am not root 2024-05-08T21:47:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.341+0000 7f11fa60a700 1 Processor -- start 2024-05-08T21:47:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11fa60a700 1 -- start start 2024-05-08T21:47:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11fa60a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4105ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11fa60a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f41060a0 con 0x7f11f41056b0 2024-05-08T21:47:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4105ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4105ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46940/0 (socket says 172.21.15.64:46940) 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11f3fff700 1 -- 172.21.15.64:0/3401888976 learned_addr learned my addr 172.21.15.64:0/3401888976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11f3fff700 1 -- 172.21.15.64:0/3401888976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f41068b0 con 0x7f11f41056b0 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.342+0000 7f11f3fff700 1 --2- 172.21.15.64:0/3401888976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4105ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f11e400bb40 tx=0x7f11e400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50406e8e2b707d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11f2ffd700 1 -- 172.21.15.64:0/3401888976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e400dbe0 con 0x7f11f41056b0 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11f2ffd700 1 -- 172.21.15.64:0/3401888976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11e40044d0 con 0x7f11f41056b0 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11f2ffd700 1 -- 172.21.15.64:0/3401888976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e4010470 con 0x7f11f41056b0 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 -- 172.21.15.64:0/3401888976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 msgr2=0x7f11f4105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 --2- 172.21.15.64:0/3401888976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4105ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f11e400bb40 tx=0x7f11e400be50 comp rx=0 tx=0).stop 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 -- 172.21.15.64:0/3401888976 shutdown_connections 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 --2- 172.21.15.64:0/3401888976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4105ad0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 -- 172.21.15.64:0/3401888976 >> 172.21.15.64:0/3401888976 conn(0x7f11f4100cd0 msgr2=0x7f11f41030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 -- 172.21.15.64:0/3401888976 shutdown_connections 2024-05-08T21:47:53.030 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.343+0000 7f11fa60a700 1 -- 172.21.15.64:0/3401888976 wait complete. 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.344+0000 7f11fa60a700 1 Processor -- start 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.344+0000 7f11fa60a700 1 -- start start 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11fa60a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11fa60a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f41988d0 con 0x7f11f41056b0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46954/0 (socket says 172.21.15.64:46954) 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f3fff700 1 -- 172.21.15.64:0/4189523467 learned_addr learned my addr 172.21.15.64:0/4189523467 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f3fff700 1 -- 172.21.15.64:0/4189523467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e400b7f0 con 0x7f11f41056b0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f3fff700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4198390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11e4003c60 tx=0x7f11e4003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e401e070 con 0x7f11f41056b0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11e4004300 con 0x7f11f41056b0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f4198ad0 con 0x7f11f41056b0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11e4022a50 con 0x7f11f41056b0 2024-05-08T21:47:53.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.345+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f4198f70 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.346+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f11e402c440 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.346+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f4078ff0 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.347+0000 7f11f17fa700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f11dc038700 0x7f11dc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.347+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f11e404eac0 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.347+0000 7f11f37fe700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f11dc038700 0x7f11dc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.347+0000 7f11f37fe700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f11dc038700 0x7f11dc03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006fd0 tx=0x7f11e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.351+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f11e402a770 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.613+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f11f4191ad0 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.666+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f11e402cd00 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.667+0000 7f11f17fa700 1 -- 172.21.15.64:0/4189523467 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f11e404e820 con 0x7f11f41056b0 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.670+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f11dc038700 msgr2=0x7f11dc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.670+0000 7f11fa60a700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f11dc038700 0x7f11dc03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006fd0 tx=0x7f11e8006e40 comp rx=0 tx=0).stop 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.670+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 msgr2=0x7f11f4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:53.032 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.670+0000 7f11fa60a700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4198390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11e4003c60 tx=0x7f11e4003980 comp rx=0 tx=0).stop 2024-05-08T21:47:53.033 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.671+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 shutdown_connections 2024-05-08T21:47:53.033 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.671+0000 7f11fa60a700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f11dc038700 0x7f11dc03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.033 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.671+0000 7f11fa60a700 1 --2- 172.21.15.64:0/4189523467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11f41056b0 0x7f11f4198390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.033 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.671+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 >> 172.21.15.64:0/4189523467 conn(0x7f11f4100cd0 msgr2=0x7f11f418eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:53.033 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.672+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 shutdown_connections 2024-05-08T21:47:53.033 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:52.672+0000 7f11fa60a700 1 -- 172.21.15.64:0/4189523467 wait complete. 2024-05-08T21:47:53.321 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:53 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:53.128+0000 7faf97d10200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:47:53.321 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:53 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:53.227+0000 7faf97d10200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:47:53.321 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:53 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:53.320+0000 7faf97d10200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:47:53.644 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:53 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:53.412+0000 7faf97d10200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:47:53.909 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:53 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/4189523467' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T21:47:53.909 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:53 smithi064 ceph-mon[42500]: mgrmap e5: x(active, since 11s) 2024-05-08T21:47:53.909 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:53 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2751509988' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T21:47:53.973 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:47:53.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T21:47:53.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:47:53.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T21:47:53.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T21:47:53.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.349+0000 7f8869d25700 1 Processor -- start 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f8869d25700 1 -- start start 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f8869d25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864109660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f8869d25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88640747f0 con 0x7f8864107270 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f88637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f88637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46992/0 (socket says 172.21.15.64:46992) 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f88637fe700 1 -- 172.21.15.64:0/2478277517 learned_addr learned my addr 172.21.15.64:0/2478277517 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f88637fe700 1 -- 172.21.15.64:0/2478277517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864109ba0 con 0x7f8864107270 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.350+0000 7f88637fe700 1 --2- 172.21.15.64:0/2478277517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864109660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88540098a0 tx=0x7f8854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4b7de59b4234d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f88627fc700 1 -- 172.21.15.64:0/2478277517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8854004160 con 0x7f8864107270 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f88627fc700 1 -- 172.21.15.64:0/2478277517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88540042c0 con 0x7f8864107270 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f88627fc700 1 -- 172.21.15.64:0/2478277517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8854011a00 con 0x7f8864107270 2024-05-08T21:47:53.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 -- 172.21.15.64:0/2478277517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 msgr2=0x7f8864109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 --2- 172.21.15.64:0/2478277517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864109660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88540098a0 tx=0x7f8854009bb0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 -- 172.21.15.64:0/2478277517 shutdown_connections 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 --2- 172.21.15.64:0/2478277517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864109660 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 -- 172.21.15.64:0/2478277517 >> 172.21.15.64:0/2478277517 conn(0x7f8864100c40 msgr2=0x7f8864103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 -- 172.21.15.64:0/2478277517 shutdown_connections 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.351+0000 7f8869d25700 1 -- 172.21.15.64:0/2478277517 wait complete. 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.352+0000 7f8869d25700 1 Processor -- start 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.352+0000 7f8869d25700 1 -- start start 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.352+0000 7f8869d25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.352+0000 7f8869d25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8864106fe0 con 0x7f8864107270 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f88637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f88637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47006/0 (socket says 172.21.15.64:47006) 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f88637fe700 1 -- 172.21.15.64:0/2751509988 learned_addr learned my addr 172.21.15.64:0/2751509988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f88637fe700 1 -- 172.21.15.64:0/2751509988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854009550 con 0x7f8864107270 2024-05-08T21:47:53.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f88637fe700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864106aa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8854009130 tx=0x7f8854011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88540038b0 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8864105220 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8854003a10 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8854019550 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.353+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88641056c0 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.354+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f8854019810 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.354+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886410a940 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.354+0000 7f8860ff9700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 0x7f884c03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.354+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f885404c4c0 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.354+0000 7f8862ffd700 1 -- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 msgr2=0x7f884c03ab50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.354+0000 7f8862ffd700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 0x7f884c03ab50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.359+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f885402b340 con 0x7f8864107270 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.555+0000 7f8862ffd700 1 -- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 msgr2=0x7f884c03ab50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:47:53.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.555+0000 7f8862ffd700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 0x7f884c03ab50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.619+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8864105e00 con 0x7f8864107270 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.619+0000 7f8860ff9700 1 -- 172.21.15.64:0/2751509988 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8854018b40 con 0x7f8864107270 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 msgr2=0x7f884c03ab50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 0x7f884c03ab50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 msgr2=0x7f8864106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864106aa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8854009130 tx=0x7f8854011ad0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 shutdown_connections 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f884c038690 0x7f884c03ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 --2- 172.21.15.64:0/2751509988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8864107270 0x7f8864106aa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.621+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 >> 172.21.15.64:0/2751509988 conn(0x7f8864100c40 msgr2=0x7f88641018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.622+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 shutdown_connections 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:53.622+0000 7f8869d25700 1 -- 172.21.15.64:0/2751509988 wait complete. 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-05-08T21:47:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2024-05-08T21:47:54.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.096+0000 7faf97d10200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:47:54.712 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.365+0000 7faf97d10200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:47:54.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.455+0000 7faf97d10200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:47:54.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.570+0000 7faf97d10200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:47:55.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.733+0000 7faf97d10200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:47:55.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.867+0000 7faf97d10200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:47:55.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:54 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:54.956+0000 7faf97d10200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:47:55.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:55 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:55.342+0000 7faf97d10200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:47:56.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:55 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:55.748+0000 7faf97d10200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:47:56.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:55 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:55.951+0000 7faf97d10200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:47:56.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:56 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:56.048+0000 7faf97d10200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:47:57.108 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:56 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:56.848+0000 7faf97d10200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:47:57.108 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:56 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:56.950+0000 7faf97d10200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:47:57.108 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:57 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:57.107+0000 7faf97d10200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:47:58.162 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:57 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:57.767+0000 7faf97d10200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:47:58.163 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:57 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:57.958+0000 7faf97d10200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:47:58.163 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:58 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:58.162+0000 7faf97d10200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:47:59.437 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:59.136+0000 7faf97d10200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:47:59.712 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:59.436+0000 7faf97d10200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:47:59.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:59.529+0000 7faf97d10200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:47:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:47:59.731+0000 7faf97d10200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: Active manager daemon x restarted 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: Activating manager daemon x 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: mgrmap e6: x(active, starting, since 0.00574871s) 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:48:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:48:00.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: Manager daemon x is now available 2024-05-08T21:48:00.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:00.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:47:59 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:01.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:48:01.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T21:48:01.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T21:48:01.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:48:01.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.333+0000 7f79554b0700 1 Processor -- start 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.333+0000 7f79554b0700 1 -- start start 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.333+0000 7f79554b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f7950109660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.333+0000 7f79554b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79500747f0 con 0x7f7950107270 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f7950109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f7950109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47010/0 (socket says 172.21.15.64:47010) 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794effd700 1 -- 172.21.15.64:0/1831033988 learned_addr learned my addr 172.21.15.64:0/1831033988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794effd700 1 -- 172.21.15.64:0/1831033988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7950109ba0 con 0x7f7950107270 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794effd700 1 --2- 172.21.15.64:0/1831033988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f7950109660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f79400098a0 tx=0x7f7940009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3540a1f4f78fe9be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794dffb700 1 -- 172.21.15.64:0/1831033988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7940004160 con 0x7f7950107270 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794dffb700 1 -- 172.21.15.64:0/1831033988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79400042c0 con 0x7f7950107270 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.334+0000 7f794dffb700 1 -- 172.21.15.64:0/1831033988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7940011a00 con 0x7f7950107270 2024-05-08T21:48:01.110 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 -- 172.21.15.64:0/1831033988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 msgr2=0x7f7950109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 --2- 172.21.15.64:0/1831033988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f7950109660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f79400098a0 tx=0x7f7940009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 -- 172.21.15.64:0/1831033988 shutdown_connections 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 --2- 172.21.15.64:0/1831033988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f7950109660 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 -- 172.21.15.64:0/1831033988 >> 172.21.15.64:0/1831033988 conn(0x7f7950100c40 msgr2=0x7f7950103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 -- 172.21.15.64:0/1831033988 shutdown_connections 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.335+0000 7f79554b0700 1 -- 172.21.15.64:0/1831033988 wait complete. 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f79554b0700 1 Processor -- start 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f79554b0700 1 -- start start 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f79554b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f795019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f79554b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795019aac0 con 0x7f7950107270 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f794effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f795019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f794effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f795019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47022/0 (socket says 172.21.15.64:47022) 2024-05-08T21:48:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f794effd700 1 -- 172.21.15.64:0/1973891101 learned_addr learned my addr 172.21.15.64:0/1973891101 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.336+0000 7f794effd700 1 -- 172.21.15.64:0/1973891101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940009550 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.337+0000 7f794effd700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f795019a580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7940009130 tx=0x7f7940011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.337+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79400038b0 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.337+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7940003a10 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.337+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795019acc0 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.337+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7940019550 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.337+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795019b160 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.338+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f794001a430 con 0x7f7950107270 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.338+0000 7f7937fff700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.338+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 --> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f793803b2b0 con 0x7f79380386e0 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.338+0000 7f794e7fc700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 msgr2=0x7f793803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:48:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.338+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f794004c430 con 0x7f7950107270 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.338+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.539+0000 7f794e7fc700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 msgr2=0x7f793803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.539+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.940+0000 7f794e7fc700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 msgr2=0x7f793803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:54.940+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:55.741+0000 7f794e7fc700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 msgr2=0x7f793803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:55.741+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:57.342+0000 7f794e7fc700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 msgr2=0x7f793803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1406677494 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:57.342+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T21:48:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:59.748+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f7940019990 con 0x7f7950107270 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:59.748+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 msgr2=0x7f793803aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:47:59.748+0000 7f7937fff700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.747+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f794004e1e0 con 0x7f7950107270 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.747+0000 7f7937fff700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f793803bff0 0x7f793803e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.747+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f793803b2b0 con 0x7f793803bff0 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.747+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f793803bff0 0x7f793803e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.748+0000 7f794e7fc700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f793803bff0 0x7f793803e3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7944003a10 tx=0x7f79440092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.749+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f793803b2b0 con 0x7f793803bff0 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.755+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f795010a940 con 0x7f793803bff0 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.755+0000 7f7937fff700 1 -- 172.21.15.64:0/1973891101 <== mgr.14120 v2:172.21.15.64:6800/865555054 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f795010a940 con 0x7f793803bff0 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f793803bff0 msgr2=0x7f793803e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:01.114 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f793803bff0 0x7f793803e3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7944003a10 tx=0x7f79440092b0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 msgr2=0x7f795019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f795019a580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7940009130 tx=0x7f7940011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 shutdown_connections 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f793803bff0 0x7f793803e3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7950107270 0x7f795019a580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 --2- 172.21.15.64:0/1973891101 >> [v2:172.21.15.64:6800/1406677494,v1:172.21.15.64:6801/1406677494] conn(0x7f79380386e0 0x7f793803aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 >> 172.21.15.64:0/1973891101 conn(0x7f7950100c40 msgr2=0x7f79501018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 shutdown_connections 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:00.756+0000 7f79554b0700 1 -- 172.21.15.64:0/1973891101 wait complete. 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2024-05-08T21:48:01.115 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2024-05-08T21:48:01.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:00 smithi064 ceph-mon[42500]: Found migration_current of "None". Setting to last migration. 2024-05-08T21:48:01.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:00 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:01.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:00 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:01.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:00 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:48:01.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:00 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:48:01.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:00 smithi064 ceph-mon[42500]: mgrmap e7: x(active, since 1.00535s) 2024-05-08T21:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.423+0000 7f5e1c84b700 1 Processor -- start 2024-05-08T21:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.423+0000 7f5e1c84b700 1 -- start start 2024-05-08T21:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1c84b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18105ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1c84b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e181060a0 con 0x7f5e181056b0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18105ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18105ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47110/0 (socket says 172.21.15.64:47110) 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1659c700 1 -- 172.21.15.64:0/2723106048 learned_addr learned my addr 172.21.15.64:0/2723106048 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1659c700 1 -- 172.21.15.64:0/2723106048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e181068b0 con 0x7f5e181056b0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1659c700 1 --2- 172.21.15.64:0/2723106048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18105ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098a0 tx=0x7f5e08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce690cb1ebe025ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1559a700 1 -- 172.21.15.64:0/2723106048 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e08004160 con 0x7f5e181056b0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1559a700 1 -- 172.21.15.64:0/2723106048 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e080042c0 con 0x7f5e181056b0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.424+0000 7f5e1559a700 1 -- 172.21.15.64:0/2723106048 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e08011a00 con 0x7f5e181056b0 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2723106048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 msgr2=0x7f5e18105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 --2- 172.21.15.64:0/2723106048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18105ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098a0 tx=0x7f5e08009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2723106048 shutdown_connections 2024-05-08T21:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 --2- 172.21.15.64:0/2723106048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18105ad0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2723106048 >> 172.21.15.64:0/2723106048 conn(0x7f5e18100c50 msgr2=0x7f5e18103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2723106048 shutdown_connections 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.425+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2723106048 wait complete. 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1c84b700 1 Processor -- start 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1c84b700 1 -- start start 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1c84b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18190710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1c84b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18190c50 con 0x7f5e181056b0 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18190710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18190710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47120/0 (socket says 172.21.15.64:47120) 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1659c700 1 -- 172.21.15.64:0/2559390580 learned_addr learned my addr 172.21.15.64:0/2559390580 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.426+0000 7f5e1659c700 1 -- 172.21.15.64:0/2559390580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e08009550 con 0x7f5e181056b0 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.427+0000 7f5e1659c700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18190710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009fb0 tx=0x7f5e08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.427+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e08003a40 con 0x7f5e181056b0 2024-05-08T21:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.427+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e08003ba0 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.427+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1818ee90 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.427+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e0801a750 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.427+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1818f330 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.428+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f5e0801a9d0 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.428+0000 7f5e077fe700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5e00038590 0x7f5e0003aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.428+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5e0804c530 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.428+0000 7f5e15d9b700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5e00038590 0x7f5e0003aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.428+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1818fd40 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.429+0000 7f5e15d9b700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5e00038590 0x7f5e0003aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c006fd0 tx=0x7f5e0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.437+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5e08023390 con 0x7f5e181056b0 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.659+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e18194180 con 0x7f5e00038590 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.666+0000 7f5e077fe700 1 -- 172.21.15.64:0/2559390580 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5e18194180 con 0x7f5e00038590 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5e00038590 msgr2=0x7f5e0003aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5e00038590 0x7f5e0003aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c006fd0 tx=0x7f5e0c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 msgr2=0x7f5e18190710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18190710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009fb0 tx=0x7f5e08011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 shutdown_connections 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5e00038590 0x7f5e0003aa50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 --2- 172.21.15.64:0/2559390580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e181056b0 0x7f5e18190710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 >> 172.21.15.64:0/2559390580 conn(0x7f5e18100c50 msgr2=0x7f5e18101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 shutdown_connections 2024-05-08T21:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:01.669+0000 7f5e1c84b700 1 -- 172.21.15.64:0/2559390580 wait complete. 2024-05-08T21:48:02.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: [08/May/2024:21:48:01] ENGINE Bus STARTING 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: [08/May/2024:21:48:01] ENGINE Serving on http://172.21.15.64:8765 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:01 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:02.928 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T21:48:02.928 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1c57a700 1 Processor -- start 2024-05-08T21:48:02.928 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1c57a700 1 -- start start 2024-05-08T21:48:02.928 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1c57a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b14106be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:02.928 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1c57a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141071b0 con 0x7f7b141067c0 2024-05-08T21:48:02.928 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1a316700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b14106be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1a316700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b14106be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41016/0 (socket says 172.21.15.64:41016) 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.356+0000 7f7b1a316700 1 -- 172.21.15.64:0/2512324279 learned_addr learned my addr 172.21.15.64:0/2512324279 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.357+0000 7f7b1a316700 1 -- 172.21.15.64:0/2512324279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b141079c0 con 0x7f7b141067c0 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.357+0000 7f7b1a316700 1 --2- 172.21.15.64:0/2512324279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b14106be0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b100098a0 tx=0x7f7b10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd11a4f22e859b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.357+0000 7f7b19314700 1 -- 172.21.15.64:0/2512324279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b10004160 con 0x7f7b141067c0 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.357+0000 7f7b19314700 1 -- 172.21.15.64:0/2512324279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b100042c0 con 0x7f7b141067c0 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.357+0000 7f7b19314700 1 -- 172.21.15.64:0/2512324279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b10011a00 con 0x7f7b141067c0 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 -- 172.21.15.64:0/2512324279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 msgr2=0x7f7b14106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 --2- 172.21.15.64:0/2512324279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b14106be0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b100098a0 tx=0x7f7b10009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 -- 172.21.15.64:0/2512324279 shutdown_connections 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 --2- 172.21.15.64:0/2512324279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b14106be0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 -- 172.21.15.64:0/2512324279 >> 172.21.15.64:0/2512324279 conn(0x7f7b14101d40 msgr2=0x7f7b141041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 -- 172.21.15.64:0/2512324279 shutdown_connections 2024-05-08T21:48:02.929 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 -- 172.21.15.64:0/2512324279 wait complete. 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 Processor -- start 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 -- start start 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.358+0000 7f7b1c57a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b1419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1c57a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1419ab10 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1a316700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b1419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1a316700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b1419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41022/0 (socket says 172.21.15.64:41022) 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1a316700 1 -- 172.21.15.64:0/1580862216 learned_addr learned my addr 172.21.15.64:0/1580862216 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1a316700 1 -- 172.21.15.64:0/1580862216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b10009550 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1a316700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b1419a5d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009130 tx=0x7f7b10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b100038b0 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b10003a10 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1419ad10 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b10019550 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.359+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1419b1b0 con 0x7f7b141067c0 2024-05-08T21:48:02.930 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.360+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f7b1001a430 con 0x7f7b141067c0 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.360+0000 7f7b0b7fe700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f7b00038570 0x7f7b0003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.360+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1410abf0 con 0x7f7b141067c0 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.360+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b1004c620 con 0x7f7b141067c0 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.361+0000 7f7b19b15700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f7b00038570 0x7f7b0003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.361+0000 7f7b19b15700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f7b00038570 0x7f7b0003aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b04006fd0 tx=0x7f7b04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.365+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b1001a730 con 0x7f7b141067c0 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.567+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b14103160 con 0x7f7b00038570 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.568+0000 7f7b0b7fe700 1 -- 172.21.15.64:0/1580862216 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7b14103160 con 0x7f7b00038570 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f7b00038570 msgr2=0x7f7b0003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f7b00038570 0x7f7b0003aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b04006fd0 tx=0x7f7b04006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 msgr2=0x7f7b1419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:02.931 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b1419a5d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009130 tx=0x7f7b10011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 shutdown_connections 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f7b00038570 0x7f7b0003aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 --2- 172.21.15.64:0/1580862216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141067c0 0x7f7b1419a5d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 >> 172.21.15.64:0/1580862216 conn(0x7f7b14101d40 msgr2=0x7f7b14102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 shutdown_connections 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:02.570+0000 7f7b1c57a700 1 -- 172.21.15.64:0/1580862216 wait complete. 2024-05-08T21:48:02.932 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:03 smithi064 ceph-mon[42500]: [08/May/2024:21:48:01] ENGINE Serving on https://172.21.15.64:7150 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:03 smithi064 ceph-mon[42500]: [08/May/2024:21:48:01] ENGINE Bus STARTED 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:03 smithi064 ceph-mon[42500]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:03 smithi064 ceph-mon[42500]: mgrmap e8: x(active, since 2s) 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:03 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:03 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:03.908 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: Generating public/private rsa key pair. 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: Your identification has been saved in /tmp/tmph0x77hd4/key. 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: Your public key has been saved in /tmp/tmph0x77hd4/key.pub. 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: The key fingerprint is: 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: SHA256:KyTrTrSQjaTsYziwOdQlqL1wWLLidk/O17HqbskIjPQ ceph-71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: The key's randomart image is: 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: +---[RSA 3072]----+ 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: | | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: | . | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: |..o. . | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: |oX.+o | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: |XoX.+ . S | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: |B* E = o | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: |**..=.+ + o | 2024-05-08T21:48:03.909 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: |oooo=. * o | 2024-05-08T21:48:03.910 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: | .o+=+. | 2024-05-08T21:48:03.910 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: +----[SHA256]-----+ 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.263+0000 7f06aaa5b700 1 Processor -- start 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.263+0000 7f06aaa5b700 1 -- start start 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.263+0000 7f06aaa5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a4106bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.263+0000 7f06aaa5b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a41071c0 con 0x7f06a41067d0 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a4106bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a4106bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41026/0 (socket says 172.21.15.64:41026) 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a3fff700 1 -- 172.21.15.64:0/934660080 learned_addr learned my addr 172.21.15.64:0/934660080 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a3fff700 1 -- 172.21.15.64:0/934660080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a41079d0 con 0x7f06a41067d0 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a3fff700 1 --2- 172.21.15.64:0/934660080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a4106bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f06940098a0 tx=0x7f0694009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fde1e6fbfddf8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a2ffd700 1 -- 172.21.15.64:0/934660080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0694004160 con 0x7f06a41067d0 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a2ffd700 1 -- 172.21.15.64:0/934660080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06940042c0 con 0x7f06a41067d0 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.264+0000 7f06a2ffd700 1 -- 172.21.15.64:0/934660080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0694011a00 con 0x7f06a41067d0 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 -- 172.21.15.64:0/934660080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 msgr2=0x7f06a4106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 --2- 172.21.15.64:0/934660080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a4106bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f06940098a0 tx=0x7f0694009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 -- 172.21.15.64:0/934660080 shutdown_connections 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 --2- 172.21.15.64:0/934660080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a4106bf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 -- 172.21.15.64:0/934660080 >> 172.21.15.64:0/934660080 conn(0x7f06a4101d90 msgr2=0x7f06a41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 -- 172.21.15.64:0/934660080 shutdown_connections 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 -- 172.21.15.64:0/934660080 wait complete. 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 Processor -- start 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.265+0000 7f06aaa5b700 1 -- start start 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06aaa5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:03.995 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06aaa5b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419cce0 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41038/0 (socket says 172.21.15.64:41038) 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a3fff700 1 -- 172.21.15.64:0/719798538 learned_addr learned my addr 172.21.15.64:0/719798538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a3fff700 1 -- 172.21.15.64:0/719798538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009550 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a3fff700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a419c7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0694009130 tx=0x7f0694011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0694003ab0 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0694003c10 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a419cee0 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.266+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06940247d0 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.267+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a419d380 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.267+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f069401abb0 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.267+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a410ac00 con 0x7f06a41067d0 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.268+0000 7f06a17fa700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0690038750 0x7f069003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:03.996 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.268+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f069404c7a0 con 0x7f06a41067d0 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.268+0000 7f06a37fe700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0690038750 0x7f069003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.268+0000 7f06a37fe700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0690038750 0x7f069003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f068c006fd0 tx=0x7f068c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.272+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0694018900 con 0x7f06a41067d0 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.477+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f06a4102fe0 con 0x7f0690038750 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.650+0000 7f06a17fa700 1 -- 172.21.15.64:0/719798538 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f06a4102fe0 con 0x7f0690038750 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.652+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0690038750 msgr2=0x7f069003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.652+0000 7f06aaa5b700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0690038750 0x7f069003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f068c006fd0 tx=0x7f068c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.652+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 msgr2=0x7f06a419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.652+0000 7f06aaa5b700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a419c7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0694009130 tx=0x7f0694011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.653+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 shutdown_connections 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.653+0000 7f06aaa5b700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0690038750 0x7f069003ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.653+0000 7f06aaa5b700 1 --2- 172.21.15.64:0/719798538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41067d0 0x7f06a419c7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.653+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 >> 172.21.15.64:0/719798538 conn(0x7f06a4101d90 msgr2=0x7f06a41028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.653+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 shutdown_connections 2024-05-08T21:48:03.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:03.653+0000 7f06aaa5b700 1 -- 172.21.15.64:0/719798538 wait complete. 2024-05-08T21:48:04.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:04 smithi064 ceph-mon[42500]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:04.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:04 smithi064 ceph-mon[42500]: Generating ssh key... 2024-05-08T21:48:04.910 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:04.910 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.348+0000 7fc71f744700 1 Processor -- start 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.348+0000 7fc71f744700 1 -- start start 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.348+0000 7fc71f744700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc71807a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.348+0000 7fc71f744700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71807a930 con 0x7fc71807beb0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc71d4e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc71807a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc71d4e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc71807a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41054/0 (socket says 172.21.15.64:41054) 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc71d4e0700 1 -- 172.21.15.64:0/2114819967 learned_addr learned my addr 172.21.15.64:0/2114819967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc71d4e0700 1 -- 172.21.15.64:0/2114819967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71807aa70 con 0x7fc71807beb0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc71d4e0700 1 --2- 172.21.15.64:0/2114819967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc71807a360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7080098a0 tx=0x7fc708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1949055bb04be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc70ffff700 1 -- 172.21.15.64:0/2114819967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc708004160 con 0x7fc71807beb0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc70ffff700 1 -- 172.21.15.64:0/2114819967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7080042c0 con 0x7fc71807beb0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.349+0000 7fc70ffff700 1 -- 172.21.15.64:0/2114819967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc708011a00 con 0x7fc71807beb0 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 -- 172.21.15.64:0/2114819967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 msgr2=0x7fc71807a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:04.911 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 --2- 172.21.15.64:0/2114819967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc71807a360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7080098a0 tx=0x7fc708009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 -- 172.21.15.64:0/2114819967 shutdown_connections 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 --2- 172.21.15.64:0/2114819967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc71807a360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 -- 172.21.15.64:0/2114819967 >> 172.21.15.64:0/2114819967 conn(0x7fc718101d40 msgr2=0x7fc7181041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 -- 172.21.15.64:0/2114819967 shutdown_connections 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 -- 172.21.15.64:0/2114819967 wait complete. 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 Processor -- start 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.350+0000 7fc71f744700 1 -- start start 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71f744700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc7181a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71f744700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181a1080 con 0x7fc71807beb0 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71d4e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc7181a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71d4e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc7181a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41064/0 (socket says 172.21.15.64:41064) 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71d4e0700 1 -- 172.21.15.64:0/607112041 learned_addr learned my addr 172.21.15.64:0/607112041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71d4e0700 1 -- 172.21.15.64:0/607112041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708009550 con 0x7fc71807beb0 2024-05-08T21:48:04.912 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71d4e0700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc7181a0b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc708011710 tx=0x7fc708011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7080039e0 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc708003b40 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7181a1280 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc708019690 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.351+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7181a1720 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.352+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc718108760 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.353+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fc70801aa60 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.353+0000 7fc70e7fc700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fc704038750 0x7fc70403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.353+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc70804c7d0 con 0x7fc71807beb0 2024-05-08T21:48:04.913 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.353+0000 7fc71ccdf700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fc704038750 0x7fc70403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.353+0000 7fc71ccdf700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fc704038750 0x7fc70403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc714006fd0 tx=0x7fc714006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.358+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc70802db40 con 0x7fc71807beb0 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.561+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7180613e0 con 0x7fc704038750 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.562+0000 7fc70e7fc700 1 -- 172.21.15.64:0/607112041 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fc7180613e0 con 0x7fc704038750 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fc704038750 msgr2=0x7fc70403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fc704038750 0x7fc70403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc714006fd0 tx=0x7fc714006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 msgr2=0x7fc7181a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc7181a0b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc708011710 tx=0x7fc708011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 shutdown_connections 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fc704038750 0x7fc70403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 --2- 172.21.15.64:0/607112041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc71807beb0 0x7fc7181a0b40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 >> 172.21.15.64:0/607112041 conn(0x7fc718101d40 msgr2=0x7fc7181041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:04.914 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 shutdown_connections 2024-05-08T21:48:04.915 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:04.564+0000 7fc71f744700 1 -- 172.21.15.64:0/607112041 wait complete. 2024-05-08T21:48:04.915 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T21:48:04.915 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T21:48:04.915 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2024-05-08T21:48:05.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:05 smithi064 ceph-mon[42500]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:06.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:06 smithi064 ceph-mon[42500]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:07.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:07 smithi064 ceph-mon[42500]: Deploying cephadm binary to smithi064 2024-05-08T21:48:07.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:07 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:08.011 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f34136700 1 Processor -- start 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f34136700 1 -- start start 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f34136700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f34136700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c07a820 con 0x7f5f2c07be30 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f31ed2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f31ed2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41072/0 (socket says 172.21.15.64:41072) 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.232+0000 7f5f31ed2700 1 -- 172.21.15.64:0/2096176793 learned_addr learned my addr 172.21.15.64:0/2096176793 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f31ed2700 1 -- 172.21.15.64:0/2096176793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c07a960 con 0x7f5f2c07be30 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f31ed2700 1 --2- 172.21.15.64:0/2096176793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c07a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f280098a0 tx=0x7f5f28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb849e5c5193058 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f30ed0700 1 -- 172.21.15.64:0/2096176793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f28004160 con 0x7f5f2c07be30 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f30ed0700 1 -- 172.21.15.64:0/2096176793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f280042c0 con 0x7f5f2c07be30 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f30ed0700 1 -- 172.21.15.64:0/2096176793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f28011a00 con 0x7f5f2c07be30 2024-05-08T21:48:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f34136700 1 -- 172.21.15.64:0/2096176793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 msgr2=0x7f5f2c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f34136700 1 --2- 172.21.15.64:0/2096176793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c07a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f280098a0 tx=0x7f5f28009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f34136700 1 -- 172.21.15.64:0/2096176793 shutdown_connections 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f34136700 1 --2- 172.21.15.64:0/2096176793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c07a2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.233+0000 7f5f34136700 1 -- 172.21.15.64:0/2096176793 >> 172.21.15.64:0/2096176793 conn(0x7f5f2c101dd0 msgr2=0x7f5f2c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.234+0000 7f5f34136700 1 -- 172.21.15.64:0/2096176793 shutdown_connections 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.234+0000 7f5f34136700 1 -- 172.21.15.64:0/2096176793 wait complete. 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.234+0000 7f5f34136700 1 Processor -- start 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.234+0000 7f5f34136700 1 -- start start 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f34136700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f34136700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c19cc80 con 0x7f5f2c07be30 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f31ed2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f31ed2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41074/0 (socket says 172.21.15.64:41074) 2024-05-08T21:48:08.013 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f31ed2700 1 -- 172.21.15.64:0/1590980295 learned_addr learned my addr 172.21.15.64:0/1590980295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f31ed2700 1 -- 172.21.15.64:0/1590980295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009550 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f31ed2700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c19c740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009130 tx=0x7f5f28011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f280038b0 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f28003a10 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f28019550 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.235+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c19ce80 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.236+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c19d320 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.236+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5f2801a960 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.236+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c10cb40 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.237+0000 7f5f22ffd700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5f180386e0 0x7f5f1803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.237+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5f2804c790 con 0x7f5f2c07be30 2024-05-08T21:48:08.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.237+0000 7f5f316d1700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5f180386e0 0x7f5f1803aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.237+0000 7f5f316d1700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5f180386e0 0x7f5f1803aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c006fd0 tx=0x7f5f1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.242+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5f28018740 con 0x7f5f2c07be30 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:05.447+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f2c1030a0 con 0x7f5f180386e0 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.658+0000 7f5f22ffd700 1 -- 172.21.15.64:0/1590980295 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f5f2c1030a0 con 0x7f5f180386e0 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5f180386e0 msgr2=0x7f5f1803aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5f180386e0 0x7f5f1803aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c006fd0 tx=0x7f5f1c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 msgr2=0x7f5f2c19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c19c740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009130 tx=0x7f5f28011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 shutdown_connections 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f5f180386e0 0x7f5f1803aba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 --2- 172.21.15.64:0/1590980295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f2c07be30 0x7f5f2c19c740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 >> 172.21.15.64:0/1590980295 conn(0x7f5f2c101dd0 msgr2=0x7f5f2c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 shutdown_connections 2024-05-08T21:48:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:07.660+0000 7f5f34136700 1 -- 172.21.15.64:0/1590980295 wait complete. 2024-05-08T21:48:08.016 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2024-05-08T21:48:08.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:08 smithi064 ceph-mon[42500]: Added host smithi064 2024-05-08T21:48:08.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:08 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:08.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:08 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f4321349700 1 Processor -- start 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f4321349700 1 -- start start 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f4321349700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c106c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f4321349700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c107200 con 0x7f431c106810 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f431affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c106c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f431affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c106c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41086/0 (socket says 172.21.15.64:41086) 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f431affd700 1 -- 172.21.15.64:0/897903954 learned_addr learned my addr 172.21.15.64:0/897903954 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:09.102 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.385+0000 7f431affd700 1 -- 172.21.15.64:0/897903954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c1079c0 con 0x7f431c106810 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.386+0000 7f431affd700 1 --2- 172.21.15.64:0/897903954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c106c30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f430c0098a0 tx=0x7f430c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee66c379ffbfb5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.386+0000 7f4319ffb700 1 -- 172.21.15.64:0/897903954 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f430c004160 con 0x7f431c106810 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.386+0000 7f4319ffb700 1 -- 172.21.15.64:0/897903954 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f430c0042c0 con 0x7f431c106810 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.386+0000 7f4319ffb700 1 -- 172.21.15.64:0/897903954 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f430c011a00 con 0x7f431c106810 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.386+0000 7f4321349700 1 -- 172.21.15.64:0/897903954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 msgr2=0x7f431c106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.386+0000 7f4321349700 1 --2- 172.21.15.64:0/897903954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c106c30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f430c0098a0 tx=0x7f430c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 -- 172.21.15.64:0/897903954 shutdown_connections 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 --2- 172.21.15.64:0/897903954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c106c30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 -- 172.21.15.64:0/897903954 >> 172.21.15.64:0/897903954 conn(0x7f431c101dd0 msgr2=0x7f431c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 -- 172.21.15.64:0/897903954 shutdown_connections 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 -- 172.21.15.64:0/897903954 wait complete. 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 Processor -- start 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 -- start start 2024-05-08T21:48:09.103 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.387+0000 7f4321349700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f4321349700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c19aaa0 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f431affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f431affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41096/0 (socket says 172.21.15.64:41096) 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f431affd700 1 -- 172.21.15.64:0/3520788934 learned_addr learned my addr 172.21.15.64:0/3520788934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f431affd700 1 -- 172.21.15.64:0/3520788934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c009550 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f431affd700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c19a560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f430c009130 tx=0x7f430c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f430c0038b0 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c19aca0 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f430c003a10 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f430c019550 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.388+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c19b140 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.389+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f430c019810 con 0x7f431c106810 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.389+0000 7f4303fff700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f4304038690 0x7f430403ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:09.104 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.389+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c10abf0 con 0x7f431c106810 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.390+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f430c04c8e0 con 0x7f431c106810 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.390+0000 7f431a7fc700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f4304038690 0x7f430403ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.390+0000 7f431a7fc700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f4304038690 0x7f430403ab50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4310006fd0 tx=0x7f4310006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.395+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f430c018b80 con 0x7f431c106810 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.619+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f431c1030e0 con 0x7f4304038690 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.624+0000 7f4303fff700 1 -- 172.21.15.64:0/3520788934 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f431c1030e0 con 0x7f4304038690 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f4304038690 msgr2=0x7f430403ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f4304038690 0x7f430403ab50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4310006fd0 tx=0x7f4310006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 msgr2=0x7f431c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c19a560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f430c009130 tx=0x7f430c011ad0 comp rx=0 tx=0).stop 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 shutdown_connections 2024-05-08T21:48:09.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f4304038690 0x7f430403ab50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:09.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 --2- 172.21.15.64:0/3520788934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f431c106810 0x7f431c19a560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:09.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 >> 172.21.15.64:0/3520788934 conn(0x7f431c101dd0 msgr2=0x7f431c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:09.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 shutdown_connections 2024-05-08T21:48:09.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:08.627+0000 7f4321349700 1 -- 172.21.15.64:0/3520788934 wait complete. 2024-05-08T21:48:09.106 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2024-05-08T21:48:09.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:09 smithi064 ceph-mon[42500]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:09.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:09 smithi064 ceph-mon[42500]: Saving service mon spec with placement count:5 2024-05-08T21:48:09.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:09 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:09.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:09 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd476ecc700 1 Processor -- start 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd476ecc700 1 -- start start 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd476ecc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd476ecc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47007a8f0 con 0x7fd47007be70 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd474c68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd474c68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41104/0 (socket says 172.21.15.64:41104) 2024-05-08T21:48:10.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.445+0000 7fd474c68700 1 -- 172.21.15.64:0/2709202941 learned_addr learned my addr 172.21.15.64:0/2709202941 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.446+0000 7fd474c68700 1 -- 172.21.15.64:0/2709202941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47007aa30 con 0x7fd47007be70 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.446+0000 7fd474c68700 1 --2- 172.21.15.64:0/2709202941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47007a320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4640098a0 tx=0x7fd464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576f18959fb2fbff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.446+0000 7fd46f7fe700 1 -- 172.21.15.64:0/2709202941 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd464004160 con 0x7fd47007be70 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.446+0000 7fd46f7fe700 1 -- 172.21.15.64:0/2709202941 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4640042c0 con 0x7fd47007be70 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.446+0000 7fd46f7fe700 1 -- 172.21.15.64:0/2709202941 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4640119b0 con 0x7fd47007be70 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 -- 172.21.15.64:0/2709202941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 msgr2=0x7fd47007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 --2- 172.21.15.64:0/2709202941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47007a320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4640098a0 tx=0x7fd464009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 -- 172.21.15.64:0/2709202941 shutdown_connections 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 --2- 172.21.15.64:0/2709202941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47007a320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 -- 172.21.15.64:0/2709202941 >> 172.21.15.64:0/2709202941 conn(0x7fd470101dd0 msgr2=0x7fd4701041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 -- 172.21.15.64:0/2709202941 shutdown_connections 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.447+0000 7fd476ecc700 1 -- 172.21.15.64:0/2709202941 wait complete. 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd476ecc700 1 Processor -- start 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd476ecc700 1 -- start start 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd476ecc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:10.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd476ecc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47019cc60 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd474c68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd474c68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41114/0 (socket says 172.21.15.64:41114) 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd474c68700 1 -- 172.21.15.64:0/2321846479 learned_addr learned my addr 172.21.15.64:0/2321846479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd474c68700 1 -- 172.21.15.64:0/2321846479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd464009550 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.448+0000 7fd474c68700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47019c720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd464006af0 tx=0x7fd464011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.449+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd464003a40 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.449+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd464003ba0 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.449+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47019ce60 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.449+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4640196c0 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.449+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47019d300 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.450+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fd46401aa80 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.450+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47010cbd0 con 0x7fd47007be70 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.450+0000 7fd46dffb700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd460038700 0x7fd46003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.450+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd46404c900 con 0x7fd47007be70 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.450+0000 7fd46ffff700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd460038700 0x7fd46003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.451+0000 7fd46ffff700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd460038700 0x7fd46003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd45c006fd0 tx=0x7fd45c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.455+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd46401a430 con 0x7fd47007be70 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.682+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd47019fb90 con 0x7fd460038700 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.686+0000 7fd46dffb700 1 -- 172.21.15.64:0/2321846479 <== mgr.14120 v2:172.21.15.64:6800/865555054 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd47019fb90 con 0x7fd460038700 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd460038700 msgr2=0x7fd46003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd460038700 0x7fd46003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd45c006fd0 tx=0x7fd45c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 msgr2=0x7fd47019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47019c720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd464006af0 tx=0x7fd464011a10 comp rx=0 tx=0).stop 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 shutdown_connections 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd460038700 0x7fd46003abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 --2- 172.21.15.64:0/2321846479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47007be70 0x7fd47019c720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 >> 172.21.15.64:0/2321846479 conn(0x7fd470101dd0 msgr2=0x7fd470102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 shutdown_connections 2024-05-08T21:48:10.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:09.689+0000 7fd476ecc700 1 -- 172.21.15.64:0/2321846479 wait complete. 2024-05-08T21:48:10.948 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:10 smithi064 ceph-mon[42500]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:10.949 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:10 smithi064 ceph-mon[42500]: Saving service mgr spec with placement count:2 2024-05-08T21:48:10.949 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:10 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:10.949 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:10 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:10.949 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:10 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2926357176' entity='client.admin' 2024-05-08T21:48:10.990 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.462+0000 7f012bd1f700 1 Processor -- start 2024-05-08T21:48:10.990 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.462+0000 7f012bd1f700 1 -- start start 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f012bd1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124105b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f012bd1f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241060e0 con 0x7f01241056f0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f0129abb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124105b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f0129abb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124105b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41130/0 (socket says 172.21.15.64:41130) 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f0129abb700 1 -- 172.21.15.64:0/3732906864 learned_addr learned my addr 172.21.15.64:0/3732906864 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f0129abb700 1 -- 172.21.15.64:0/3732906864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01241068f0 con 0x7f01241056f0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.463+0000 7f0129abb700 1 --2- 172.21.15.64:0/3732906864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124105b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f01200098a0 tx=0x7f0120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ec784f7fecd851 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f0128ab9700 1 -- 172.21.15.64:0/3732906864 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0120004160 con 0x7f01241056f0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f0128ab9700 1 -- 172.21.15.64:0/3732906864 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01200042c0 con 0x7f01241056f0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f0128ab9700 1 -- 172.21.15.64:0/3732906864 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01200119b0 con 0x7f01241056f0 2024-05-08T21:48:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f012bd1f700 1 -- 172.21.15.64:0/3732906864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 msgr2=0x7f0124105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f012bd1f700 1 --2- 172.21.15.64:0/3732906864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124105b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f01200098a0 tx=0x7f0120009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f012bd1f700 1 -- 172.21.15.64:0/3732906864 shutdown_connections 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f012bd1f700 1 --2- 172.21.15.64:0/3732906864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124105b10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.464+0000 7f012bd1f700 1 -- 172.21.15.64:0/3732906864 >> 172.21.15.64:0/3732906864 conn(0x7f0124100c90 msgr2=0x7f01241030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.465+0000 7f012bd1f700 1 -- 172.21.15.64:0/3732906864 shutdown_connections 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.465+0000 7f012bd1f700 1 -- 172.21.15.64:0/3732906864 wait complete. 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.465+0000 7f012bd1f700 1 Processor -- start 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.465+0000 7f012bd1f700 1 -- start start 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.465+0000 7f012bd1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124190810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.465+0000 7f012bd1f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124190d50 con 0x7f01241056f0 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f0129abb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124190810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f0129abb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124190810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41140/0 (socket says 172.21.15.64:41140) 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f0129abb700 1 -- 172.21.15.64:0/2926357176 learned_addr learned my addr 172.21.15.64:0/2926357176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f0129abb700 1 -- 172.21.15.64:0/2926357176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120009550 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f0129abb700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124190810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0120003e60 tx=0x7f0120011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01200038b0 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0120003a10 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012418ef90 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.466+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0120019530 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.467+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012418f430 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.468+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01240625d0 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.468+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f012001a980 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.468+0000 7f011affd700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0110038700 0x7f011003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.468+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f012004c710 con 0x7f01241056f0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.468+0000 7f01292ba700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0110038700 0x7f011003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.469+0000 7f01292ba700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0110038700 0x7f011003abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f0114006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.473+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0120003b80 con 0x7f01241056f0 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.678+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0124109b20 con 0x7f01241056f0 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.681+0000 7f011affd700 1 -- 172.21.15.64:0/2926357176 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f012002de00 con 0x7f01241056f0 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.683+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0110038700 msgr2=0x7f011003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.683+0000 7f012bd1f700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0110038700 0x7f011003abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f0114006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.683+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 msgr2=0x7f0124190810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.683+0000 7f012bd1f700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124190810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0120003e60 tx=0x7f0120011a30 comp rx=0 tx=0).stop 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.684+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 shutdown_connections 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.684+0000 7f012bd1f700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f0110038700 0x7f011003abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.684+0000 7f012bd1f700 1 --2- 172.21.15.64:0/2926357176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01241056f0 0x7f0124190810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.684+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 >> 172.21.15.64:0/2926357176 conn(0x7f0124100c90 msgr2=0x7f0124101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.684+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 shutdown_connections 2024-05-08T21:48:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:10.684+0000 7f012bd1f700 1 -- 172.21.15.64:0/2926357176 wait complete. 2024-05-08T21:48:12.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0bcefd700 1 Processor -- start 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0bcefd700 1 -- start start 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0bcefd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0bcefd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b807a840 con 0x7fa0b807be50 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41156/0 (socket says 172.21.15.64:41156) 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.374+0000 7fa0b659c700 1 -- 172.21.15.64:0/3690909751 learned_addr learned my addr 172.21.15.64:0/3690909751 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0b659c700 1 -- 172.21.15.64:0/3690909751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b807a980 con 0x7fa0b807be50 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0b659c700 1 --2- 172.21.15.64:0/3690909751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b807a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098a0 tx=0x7fa0a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e940281bc1156bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0b559a700 1 -- 172.21.15.64:0/3690909751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8004160 con 0x7fa0b807be50 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0b559a700 1 -- 172.21.15.64:0/3690909751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a80042c0 con 0x7fa0b807be50 2024-05-08T21:48:12.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0b559a700 1 -- 172.21.15.64:0/3690909751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a80119b0 con 0x7fa0b807be50 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0bcefd700 1 -- 172.21.15.64:0/3690909751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 msgr2=0x7fa0b807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.375+0000 7fa0bcefd700 1 --2- 172.21.15.64:0/3690909751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b807a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098a0 tx=0x7fa0a8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.376+0000 7fa0bcefd700 1 -- 172.21.15.64:0/3690909751 shutdown_connections 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.376+0000 7fa0bcefd700 1 --2- 172.21.15.64:0/3690909751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b807a300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.376+0000 7fa0bcefd700 1 -- 172.21.15.64:0/3690909751 >> 172.21.15.64:0/3690909751 conn(0x7fa0b8101da0 msgr2=0x7fa0b81041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.376+0000 7fa0bcefd700 1 -- 172.21.15.64:0/3690909751 shutdown_connections 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.376+0000 7fa0bcefd700 1 -- 172.21.15.64:0/3690909751 wait complete. 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0bcefd700 1 Processor -- start 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0bcefd700 1 -- start start 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0bcefd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0bcefd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b819ccd0 con 0x7fa0b807be50 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41166/0 (socket says 172.21.15.64:41166) 2024-05-08T21:48:12.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0b659c700 1 -- 172.21.15.64:0/2856859562 learned_addr learned my addr 172.21.15.64:0/2856859562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0b659c700 1 -- 172.21.15.64:0/2856859562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a8009550 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.377+0000 7fa0b659c700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b819c790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8003e60 tx=0x7fa0a8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.378+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8003930 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.378+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a8003a90 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.378+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b819ced0 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.378+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8019630 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.378+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b819d370 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.379+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa0a8019860 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.379+0000 7fa0a77fe700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fa0a0038700 0x7fa0a003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.379+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa0a804c830 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.379+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b810cb80 con 0x7fa0b807be50 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.379+0000 7fa0b5d9b700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fa0a0038700 0x7fa0a003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.379+0000 7fa0b5d9b700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fa0a0038700 0x7fa0a003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:12.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.384+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0a8019b60 con 0x7fa0b807be50 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.591+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa0b810cb80 con 0x7fa0b807be50 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.594+0000 7fa0a77fe700 1 -- 172.21.15.64:0/2856859562 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa0a8018b40 con 0x7fa0b807be50 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fa0a0038700 msgr2=0x7fa0a003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fa0a0038700 0x7fa0a003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 msgr2=0x7fa0b819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b819c790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8003e60 tx=0x7fa0a8011a80 comp rx=0 tx=0).stop 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 shutdown_connections 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fa0a0038700 0x7fa0a003abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.596+0000 7fa0bcefd700 1 --2- 172.21.15.64:0/2856859562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b807be50 0x7fa0b819c790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.597+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 >> 172.21.15.64:0/2856859562 conn(0x7fa0b8101da0 msgr2=0x7fa0b8102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.597+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 shutdown_connections 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:11.597+0000 7fa0bcefd700 1 -- 172.21.15.64:0/2856859562 wait complete. 2024-05-08T21:48:12.089 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2024-05-08T21:48:12.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:11 smithi064 ceph-mon[42500]: from='mgr.14120 172.21.15.64:0/1869071516' entity='mgr.x' 2024-05-08T21:48:12.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:11 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2856859562' entity='client.admin' 2024-05-08T21:48:13.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:12 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1390112776' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T21:48:13.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setuser ceph since I am not root 2024-05-08T21:48:13.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setgroup ceph since I am not root 2024-05-08T21:48:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9cb592d700 1 Processor -- start 2024-05-08T21:48:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9cb592d700 1 -- start start 2024-05-08T21:48:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9cb592d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb01054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9cb592d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb0079e70 con 0x7f9cb0105090 2024-05-08T21:48:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb01054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb01054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37512/0 (socket says 172.21.15.64:37512) 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.455+0000 7f9caeffd700 1 -- 172.21.15.64:0/506607627 learned_addr learned my addr 172.21.15.64:0/506607627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.456+0000 7f9caeffd700 1 -- 172.21.15.64:0/506607627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb01059f0 con 0x7f9cb0105090 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.456+0000 7f9caeffd700 1 --2- 172.21.15.64:0/506607627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb01054b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098a0 tx=0x7f9ca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203e1e311b687189 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.456+0000 7f9cadffb700 1 -- 172.21.15.64:0/506607627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4004160 con 0x7f9cb0105090 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.456+0000 7f9cadffb700 1 -- 172.21.15.64:0/506607627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca40042c0 con 0x7f9cb0105090 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.456+0000 7f9cadffb700 1 -- 172.21.15.64:0/506607627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca40119b0 con 0x7f9cb0105090 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 -- 172.21.15.64:0/506607627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 msgr2=0x7f9cb01054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 --2- 172.21.15.64:0/506607627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb01054b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098a0 tx=0x7f9ca4009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 -- 172.21.15.64:0/506607627 shutdown_connections 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 --2- 172.21.15.64:0/506607627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb01054b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 -- 172.21.15.64:0/506607627 >> 172.21.15.64:0/506607627 conn(0x7f9cb0100cd0 msgr2=0x7f9cb01030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 -- 172.21.15.64:0/506607627 shutdown_connections 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 -- 172.21.15.64:0/506607627 wait complete. 2024-05-08T21:48:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.457+0000 7f9cb592d700 1 Processor -- start 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9cb592d700 1 -- start start 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9cb592d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9cb592d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb019cc40 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37528/0 (socket says 172.21.15.64:37528) 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9caeffd700 1 -- 172.21.15.64:0/1390112776 learned_addr learned my addr 172.21.15.64:0/1390112776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9caeffd700 1 -- 172.21.15.64:0/1390112776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4009550 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.458+0000 7f9caeffd700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb019c700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006af0 tx=0x7f9ca4011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.459+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4003a40 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.459+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca4003ba0 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.459+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb019ce40 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.459+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca40196c0 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.459+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb019d2e0 con 0x7f9cb0105090 2024-05-08T21:48:13.304 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.460+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f9ca4019930 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.460+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb01963b0 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.460+0000 7f9cb492b700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f9c98038700 0x7f9c9803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.460+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ca404c640 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.460+0000 7f9cae7fc700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f9c98038700 0x7f9c9803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.460+0000 7f9cae7fc700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f9c98038700 0x7f9c9803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0006fd0 tx=0x7f9ca0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.465+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ca4018350 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.729+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f9cb0106a70 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.911+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f9ca402d9a0 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.912+0000 7f9cb492b700 1 -- 172.21.15.64:0/1390112776 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f9ca404c3a0 con 0x7f9cb0105090 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f9c98038700 msgr2=0x7f9c9803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f9c98038700 0x7f9c9803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0006fd0 tx=0x7f9ca0006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 msgr2=0x7f9cb019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb019c700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006af0 tx=0x7f9ca4011a10 comp rx=0 tx=0).stop 2024-05-08T21:48:13.305 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 shutdown_connections 2024-05-08T21:48:13.306 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7f9c98038700 0x7f9c9803abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:13.306 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 --2- 172.21.15.64:0/1390112776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cb0105090 0x7f9cb019c700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:13.306 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.914+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 >> 172.21.15.64:0/1390112776 conn(0x7f9cb0100cd0 msgr2=0x7f9cb01934f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:13.306 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.915+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 shutdown_connections 2024-05-08T21:48:13.306 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:12.915+0000 7f9cb592d700 1 -- 172.21.15.64:0/1390112776 wait complete. 2024-05-08T21:48:13.486 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:13.382+0000 7f0e13e0c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:48:13.486 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:13.484+0000 7f0e13e0c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:48:13.911 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:13.578+0000 7f0e13e0c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:48:13.912 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:13.669+0000 7f0e13e0c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:48:14.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:13 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1390112776' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T21:48:14.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:13 smithi064 ceph-mon[42500]: mgrmap e9: x(active, since 13s) 2024-05-08T21:48:14.279 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:48:14.286 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T21:48:14.286 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:48:14.286 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.657+0000 7fed62f03700 1 Processor -- start 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.657+0000 7fed62f03700 1 -- start start 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.657+0000 7fed62f03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.657+0000 7fed62f03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c074850 con 0x7fed5c107210 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed60c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed60c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37552/0 (socket says 172.21.15.64:37552) 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed60c9f700 1 -- 172.21.15.64:0/1067566854 learned_addr learned my addr 172.21.15.64:0/1067566854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed60c9f700 1 -- 172.21.15.64:0/1067566854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c109b40 con 0x7fed5c107210 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed60c9f700 1 --2- 172.21.15.64:0/1067566854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c109600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed500098a0 tx=0x7fed50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0a189d438427ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed5b7fe700 1 -- 172.21.15.64:0/1067566854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed50004160 con 0x7fed5c107210 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed5b7fe700 1 -- 172.21.15.64:0/1067566854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed500042c0 con 0x7fed5c107210 2024-05-08T21:48:14.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.658+0000 7fed5b7fe700 1 -- 172.21.15.64:0/1067566854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed500119b0 con 0x7fed5c107210 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 -- 172.21.15.64:0/1067566854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 msgr2=0x7fed5c109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 --2- 172.21.15.64:0/1067566854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c109600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed500098a0 tx=0x7fed50009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 -- 172.21.15.64:0/1067566854 shutdown_connections 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 --2- 172.21.15.64:0/1067566854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c109600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 -- 172.21.15.64:0/1067566854 >> 172.21.15.64:0/1067566854 conn(0x7fed5c100c40 msgr2=0x7fed5c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 -- 172.21.15.64:0/1067566854 shutdown_connections 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.659+0000 7fed62f03700 1 -- 172.21.15.64:0/1067566854 wait complete. 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed62f03700 1 Processor -- start 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed62f03700 1 -- start start 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed62f03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed62f03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c106f80 con 0x7fed5c107210 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed60c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:14.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed60c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37556/0 (socket says 172.21.15.64:37556) 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.660+0000 7fed60c9f700 1 -- 172.21.15.64:0/365496036 learned_addr learned my addr 172.21.15.64:0/365496036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed60c9f700 1 -- 172.21.15.64:0/365496036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50009550 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed60c9f700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c106a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed50003e60 tx=0x7fed50011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed500038b0 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c1051c0 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed50003a10 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed50019530 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.661+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c105660 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.662+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fed500197a0 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.662+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5c10a980 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.662+0000 7fed59ffb700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 0x7fed4c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.662+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed5004c800 con 0x7fed5c107210 2024-05-08T21:48:14.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.662+0000 7fed5bfff700 1 -- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 msgr2=0x7fed4c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.662+0000 7fed5bfff700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 0x7fed4c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.667+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fed5002b340 con 0x7fed5c107210 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.863+0000 7fed5bfff700 1 -- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 msgr2=0x7fed4c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.863+0000 7fed5bfff700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 0x7fed4c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.932+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fed5c105da0 con 0x7fed5c107210 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.932+0000 7fed59ffb700 1 -- 172.21.15.64:0/365496036 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fed50018b40 con 0x7fed5c107210 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 msgr2=0x7fed4c03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 0x7fed4c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 msgr2=0x7fed5c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c106a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed50003e60 tx=0x7fed50011a30 comp rx=0 tx=0).stop 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 shutdown_connections 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fed4c038750 0x7fed4c03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:14.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.934+0000 7fed62f03700 1 --2- 172.21.15.64:0/365496036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c107210 0x7fed5c106a40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:14.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.935+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 >> 172.21.15.64:0/365496036 conn(0x7fed5c100c40 msgr2=0x7fed5c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:14.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.935+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 shutdown_connections 2024-05-08T21:48:14.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:13.935+0000 7fed62f03700 1 -- 172.21.15.64:0/365496036 wait complete. 2024-05-08T21:48:14.291 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-05-08T21:48:14.291 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2024-05-08T21:48:14.622 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:14.348+0000 7f0e13e0c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:48:14.912 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:14.620+0000 7f0e13e0c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:48:14.912 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:14.711+0000 7f0e13e0c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:48:14.912 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:14.826+0000 7f0e13e0c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:48:15.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:14 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/365496036' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T21:48:15.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:14.991+0000 7f0e13e0c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:48:15.213 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:15.124+0000 7f0e13e0c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:48:15.603 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:15.214+0000 7f0e13e0c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:48:15.962 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:15.602+0000 7f0e13e0c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:48:16.313 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:16.010+0000 7f0e13e0c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:48:16.314 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:16.214+0000 7f0e13e0c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:48:17.374 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:16.312+0000 7f0e13e0c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:48:17.374 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:17.114+0000 7f0e13e0c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:48:17.374 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:17.215+0000 7f0e13e0c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:48:17.374 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:17.372+0000 7f0e13e0c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:48:18.435 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:18.034+0000 7f0e13e0c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:48:18.435 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:18.226+0000 7f0e13e0c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:48:19.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:18.433+0000 7f0e13e0c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:48:19.713 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:19.414+0000 7f0e13e0c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:48:20.021 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:19.721+0000 7f0e13e0c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:48:20.021 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:19.815+0000 7f0e13e0c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:48:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:48:20.020+0000 7f0e13e0c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: Active manager daemon x restarted 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: Activating manager daemon x 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: mgrmap e10: x(active, starting, since 0.00346786s) 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:48:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:48:20.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: Manager daemon x is now available 2024-05-08T21:48:20.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:21.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:48:21.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:48:21.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: [08/May/2024:21:48:20] ENGINE Bus STARTING 2024-05-08T21:48:21.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: [08/May/2024:21:48:20] ENGINE Serving on http://172.21.15.64:8765 2024-05-08T21:48:21.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: [08/May/2024:21:48:20] ENGINE Serving on https://172.21.15.64:7150 2024-05-08T21:48:21.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: [08/May/2024:21:48:20] ENGINE Bus STARTED 2024-05-08T21:48:21.323 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:21.323 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:21 smithi064 ceph-mon[42500]: mgrmap e11: x(active, since 1.00777s) 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.605+0000 7fd9f9919700 1 Processor -- start 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.605+0000 7fd9f9919700 1 -- start start 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.605+0000 7fd9f9919700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f4109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.605+0000 7fd9f9919700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f4074850 con 0x7fd9f4107210 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f4109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f4109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37558/0 (socket says 172.21.15.64:37558) 2024-05-08T21:48:21.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f2ffd700 1 -- 172.21.15.64:0/474983758 learned_addr learned my addr 172.21.15.64:0/474983758 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f2ffd700 1 -- 172.21.15.64:0/474983758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4109b40 con 0x7fd9f4107210 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f2ffd700 1 --2- 172.21.15.64:0/474983758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f4109600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40098a0 tx=0x7fd9e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d129269484c6999 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f1ffb700 1 -- 172.21.15.64:0/474983758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9e4004160 con 0x7fd9f4107210 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f1ffb700 1 -- 172.21.15.64:0/474983758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e40042c0 con 0x7fd9f4107210 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.606+0000 7fd9f1ffb700 1 -- 172.21.15.64:0/474983758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9e40119b0 con 0x7fd9f4107210 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 -- 172.21.15.64:0/474983758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 msgr2=0x7fd9f4109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 --2- 172.21.15.64:0/474983758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f4109600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40098a0 tx=0x7fd9e4009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 -- 172.21.15.64:0/474983758 shutdown_connections 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 --2- 172.21.15.64:0/474983758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f4109600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 -- 172.21.15.64:0/474983758 >> 172.21.15.64:0/474983758 conn(0x7fd9f4100c40 msgr2=0x7fd9f4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 -- 172.21.15.64:0/474983758 shutdown_connections 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.607+0000 7fd9f9919700 1 -- 172.21.15.64:0/474983758 wait complete. 2024-05-08T21:48:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f9919700 1 Processor -- start 2024-05-08T21:48:21.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f9919700 1 -- start start 2024-05-08T21:48:21.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f9919700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:21.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f9919700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f419aac0 con 0x7fd9f4107210 2024-05-08T21:48:21.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37568/0 (socket says 172.21.15.64:37568) 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.608+0000 7fd9f2ffd700 1 -- 172.21.15.64:0/4128587101 learned_addr learned my addr 172.21.15.64:0/4128587101 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.609+0000 7fd9f2ffd700 1 -- 172.21.15.64:0/4128587101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4009550 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.609+0000 7fd9f2ffd700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f419a580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4011980 tx=0x7fd9e4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.609+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9e4003af0 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.609+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e4003c50 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.609+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9e4019760 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.610+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f419acc0 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.610+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f419b160 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.610+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fd9e401ab10 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.611+0000 7fd9f8917700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.611+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 --> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9dc03b370 con 0x7fd9dc0387a0 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.611+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd9e404c300 con 0x7fd9f4107210 2024-05-08T21:48:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.611+0000 7fd9f27fc700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 msgr2=0x7fd9dc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.611+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.812+0000 7fd9f27fc700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 msgr2=0x7fd9dc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:14.812+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:15.213+0000 7fd9f27fc700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 msgr2=0x7fd9dc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:15.213+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:16.014+0000 7fd9f27fc700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 msgr2=0x7fd9dc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:16.014+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:17.615+0000 7fd9f27fc700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 msgr2=0x7fd9dc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/865555054 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:17.615+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:20.034+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7fd9e4019cd0 con 0x7fd9f4107210 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:20.035+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 msgr2=0x7fd9dc03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:48:21.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:20.035+0000 7fd9f8917700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.038+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fd9e4010dc0 con 0x7fd9f4107210 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.038+0000 7fd9f8917700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd9dc03bff0 0x7fd9dc03e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.038+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9dc03b370 con 0x7fd9dc03bff0 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.039+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd9dc03bff0 0x7fd9dc03e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.039+0000 7fd9f27fc700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd9dc03bff0 0x7fd9dc03e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8003a10 tx=0x7fd9e8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.040+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fd9dc03b370 con 0x7fd9dc03bff0 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.046+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd9f410a980 con 0x7fd9dc03bff0 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f8917700 1 -- 172.21.15.64:0/4128587101 <== mgr.14152 v2:172.21.15.64:6800/3946872331 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd9f410a980 con 0x7fd9dc03bff0 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd9dc03bff0 msgr2=0x7fd9dc03e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd9dc03bff0 0x7fd9dc03e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8003a10 tx=0x7fd9e8009270 comp rx=0 tx=0).stop 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 msgr2=0x7fd9f419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f419a580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4011980 tx=0x7fd9e4011a60 comp rx=0 tx=0).stop 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 shutdown_connections 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd9dc03bff0 0x7fd9dc03e3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:21.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f4107210 0x7fd9f419a580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 --2- 172.21.15.64:0/4128587101 >> [v2:172.21.15.64:6800/865555054,v1:172.21.15.64:6801/865555054] conn(0x7fd9dc0387a0 0x7fd9dc03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 >> 172.21.15.64:0/4128587101 conn(0x7fd9f4100c40 msgr2=0x7fd9f41018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 shutdown_connections 2024-05-08T21:48:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.047+0000 7fd9f9919700 1 -- 172.21.15.64:0/4128587101 wait complete. 2024-05-08T21:48:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2024-05-08T21:48:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2024-05-08T21:48:22.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:22 smithi064 ceph-mon[42500]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T21:48:22.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:22 smithi064 ceph-mon[42500]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T21:48:22.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:22.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:22.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:22.785 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T21:48:22.785 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.772+0000 7f2b626fc700 1 Processor -- start 2024-05-08T21:48:22.785 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b626fc700 1 -- start start 2024-05-08T21:48:22.785 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b626fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:22.785 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b626fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c105490 con 0x7f2b5c104aa0 2024-05-08T21:48:22.785 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c104ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37632/0 (socket says 172.21.15.64:37632) 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b5bfff700 1 -- 172.21.15.64:0/3168680841 learned_addr learned my addr 172.21.15.64:0/3168680841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.773+0000 7f2b5bfff700 1 -- 172.21.15.64:0/3168680841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c107a30 con 0x7f2b5c104aa0 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.774+0000 7f2b5bfff700 1 --2- 172.21.15.64:0/3168680841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c104ec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0098a0 tx=0x7f2b4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a50be630cc566ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.774+0000 7f2b5affd700 1 -- 172.21.15.64:0/3168680841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b4c004160 con 0x7f2b5c104aa0 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.774+0000 7f2b5affd700 1 -- 172.21.15.64:0/3168680841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b4c0042c0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.774+0000 7f2b5affd700 1 -- 172.21.15.64:0/3168680841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b4c0119b0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.774+0000 7f2b626fc700 1 -- 172.21.15.64:0/3168680841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 msgr2=0x7f2b5c104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.774+0000 7f2b626fc700 1 --2- 172.21.15.64:0/3168680841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c104ec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0098a0 tx=0x7f2b4c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 -- 172.21.15.64:0/3168680841 shutdown_connections 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 --2- 172.21.15.64:0/3168680841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c104ec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 -- 172.21.15.64:0/3168680841 >> 172.21.15.64:0/3168680841 conn(0x7f2b5c1003f0 msgr2=0x7f2b5c102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 -- 172.21.15.64:0/3168680841 shutdown_connections 2024-05-08T21:48:22.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 -- 172.21.15.64:0/3168680841 wait complete. 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 Processor -- start 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.775+0000 7f2b626fc700 1 -- start start 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b626fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b626fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c19aaf0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37640/0 (socket says 172.21.15.64:37640) 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b5bfff700 1 -- 172.21.15.64:0/1511475459 learned_addr learned my addr 172.21.15.64:0/1511475459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b5bfff700 1 -- 172.21.15.64:0/1511475459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c009550 con 0x7f2b5c104aa0 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b5bfff700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c19a5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c003e60 tx=0x7f2b4c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.776+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b4c003a60 con 0x7f2b5c104aa0 2024-05-08T21:48:22.787 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.777+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b4c003bc0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.777+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c19acf0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.777+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b4c0196c0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.777+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c19b190 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.778+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c10ac60 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.778+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f2b4c01ab60 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.779+0000 7f2b597fa700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2b44038670 0x7f2b4403ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.779+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2b4c04cad0 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.779+0000 7f2b5b7fe700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2b44038670 0x7f2b4403ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.779+0000 7f2b5b7fe700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2b44038670 0x7f2b4403ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b50006fd0 tx=0x7f2b50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:21.783+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2b4c01a430 con 0x7f2b5c104aa0 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.021+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b5c19da20 con 0x7f2b44038670 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.110+0000 7f2b597fa700 1 -- 172.21.15.64:0/1511475459 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f2b5c19da20 con 0x7f2b44038670 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2b44038670 msgr2=0x7f2b4403ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2b44038670 0x7f2b4403ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b50006fd0 tx=0x7f2b50006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:22.788 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 msgr2=0x7f2b5c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c19a5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c003e60 tx=0x7f2b4c011a30 comp rx=0 tx=0).stop 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 shutdown_connections 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2b44038670 0x7f2b4403ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 --2- 172.21.15.64:0/1511475459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b5c104aa0 0x7f2b5c19a5b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 >> 172.21.15.64:0/1511475459 conn(0x7f2b5c1003f0 msgr2=0x7f2b5c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 shutdown_connections 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:22.113+0000 7f2b626fc700 1 -- 172.21.15.64:0/1511475459 wait complete. 2024-05-08T21:48:22.789 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2024-05-08T21:48:23.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:23 smithi064 ceph-mon[42500]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:23.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:23 smithi064 ceph-mon[42500]: mgrmap e12: x(active, since 2s) 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ne1H.A0TVnKUktw6LkRa..6Vb2VGtEi.DEzYgRBDKsNAzJlaC91tW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715204903, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.133+0000 7fc9a7782700 1 Processor -- start 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.133+0000 7fc9a7782700 1 -- start start 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a7782700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a0105ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a7782700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a0106090 con 0x7fc9a01056a0 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a551e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a0105ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a551e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a0105ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42368/0 (socket says 172.21.15.64:42368) 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a551e700 1 -- 172.21.15.64:0/3951798864 learned_addr learned my addr 172.21.15.64:0/3951798864 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a551e700 1 -- 172.21.15.64:0/3951798864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a01068a0 con 0x7fc9a01056a0 2024-05-08T21:48:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc9a551e700 1 --2- 172.21.15.64:0/3951798864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a0105ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc99000bb40 tx=0x7fc99000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aec0003ec3b609b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc997fff700 1 -- 172.21.15.64:0/3951798864 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc99000dbe0 con 0x7fc9a01056a0 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc997fff700 1 -- 172.21.15.64:0/3951798864 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9900044f0 con 0x7fc9a01056a0 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.134+0000 7fc997fff700 1 -- 172.21.15.64:0/3951798864 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc990003e60 con 0x7fc9a01056a0 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 -- 172.21.15.64:0/3951798864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 msgr2=0x7fc9a0105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 --2- 172.21.15.64:0/3951798864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a0105ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc99000bb40 tx=0x7fc99000be50 comp rx=0 tx=0).stop 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 -- 172.21.15.64:0/3951798864 shutdown_connections 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 --2- 172.21.15.64:0/3951798864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a0105ac0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 -- 172.21.15.64:0/3951798864 >> 172.21.15.64:0/3951798864 conn(0x7fc9a0100c40 msgr2=0x7fc9a0103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 -- 172.21.15.64:0/3951798864 shutdown_connections 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.135+0000 7fc9a7782700 1 -- 172.21.15.64:0/3951798864 wait complete. 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a7782700 1 Processor -- start 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a7782700 1 -- start start 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a7782700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:23.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a7782700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a019cd10 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a551e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a551e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42374/0 (socket says 172.21.15.64:42374) 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a551e700 1 -- 172.21.15.64:0/1153983253 learned_addr learned my addr 172.21.15.64:0/1153983253 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a551e700 1 -- 172.21.15.64:0/1153983253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99000b7f0 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.136+0000 7fc9a551e700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a019c7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc990003bf0 tx=0x7fc990003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.137+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc990010500 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.137+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc990010b20 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.137+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a019cf10 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.137+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc990019ab0 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.137+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a019d350 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.137+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fc990019cf0 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.138+0000 7fc9967fc700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc98c038750 0x7fc98c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.138+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc99004c980 con 0x7fc9a01056a0 2024-05-08T21:48:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.138+0000 7fc9a4d1d700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc98c038750 0x7fc98c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.138+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a0073970 con 0x7fc9a01056a0 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.138+0000 7fc9a4d1d700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc98c038750 0x7fc98c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.143+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc990028940 con 0x7fc9a01056a0 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.362+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc9a0073b00 con 0x7fc98c038750 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.608+0000 7fc9967fc700 1 -- 172.21.15.64:0/1153983253 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fc9a0073b00 con 0x7fc98c038750 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc98c038750 msgr2=0x7fc98c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc98c038750 0x7fc98c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 msgr2=0x7fc9a019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a019c7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc990003bf0 tx=0x7fc990003fb0 comp rx=0 tx=0).stop 2024-05-08T21:48:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 shutdown_connections 2024-05-08T21:48:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc98c038750 0x7fc98c03ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 --2- 172.21.15.64:0/1153983253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9a01056a0 0x7fc9a019c7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 >> 172.21.15.64:0/1153983253 conn(0x7fc9a0100c40 msgr2=0x7fc9a0192ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 shutdown_connections 2024-05-08T21:48:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:23.610+0000 7fc9a7782700 1 -- 172.21.15.64:0/1153983253 wait complete. 2024-05-08T21:48:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2024-05-08T21:48:24.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:24 smithi064 ceph-mon[42500]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:24.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:24.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1357815448' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T21:48:24.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T21:48:24.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d25470700 1 Processor -- start 2024-05-08T21:48:24.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d25470700 1 -- start start 2024-05-08T21:48:24.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d25470700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d20105af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d25470700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201060c0 con 0x7f9d201056d0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d20105af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d20105af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42378/0 (socket says 172.21.15.64:42378) 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.314+0000 7f9d1effd700 1 -- 172.21.15.64:0/2653623899 learned_addr learned my addr 172.21.15.64:0/2653623899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d1effd700 1 -- 172.21.15.64:0/2653623899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d201068d0 con 0x7f9d201056d0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d1effd700 1 --2- 172.21.15.64:0/2653623899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d20105af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098a0 tx=0x7f9d10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe44f61cc9edd2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d1dffb700 1 -- 172.21.15.64:0/2653623899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d10004160 con 0x7f9d201056d0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d1dffb700 1 -- 172.21.15.64:0/2653623899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d100042c0 con 0x7f9d201056d0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d1dffb700 1 -- 172.21.15.64:0/2653623899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d100119b0 con 0x7f9d201056d0 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d25470700 1 -- 172.21.15.64:0/2653623899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 msgr2=0x7f9d20105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.315+0000 7f9d25470700 1 --2- 172.21.15.64:0/2653623899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d20105af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098a0 tx=0x7f9d10009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 -- 172.21.15.64:0/2653623899 shutdown_connections 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 --2- 172.21.15.64:0/2653623899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d20105af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 -- 172.21.15.64:0/2653623899 >> 172.21.15.64:0/2653623899 conn(0x7f9d20100cd0 msgr2=0x7f9d201030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 -- 172.21.15.64:0/2653623899 shutdown_connections 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 -- 172.21.15.64:0/2653623899 wait complete. 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 Processor -- start 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.316+0000 7f9d25470700 1 -- start start 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d25470700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d201907a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d25470700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20190ce0 con 0x7f9d201056d0 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d201907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d201907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42386/0 (socket says 172.21.15.64:42386) 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d1effd700 1 -- 172.21.15.64:0/1357815448 learned_addr learned my addr 172.21.15.64:0/1357815448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d1effd700 1 -- 172.21.15.64:0/1357815448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009550 con 0x7f9d201056d0 2024-05-08T21:48:24.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d1effd700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d201907a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d10003e60 tx=0x7f9d10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d10003a60 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d10003bc0 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2018ef70 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d100196c0 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.317+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2018f410 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.318+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f9d10019980 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.318+0000 7f9d07fff700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d08038730 0x7f9d0803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.318+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9d1004caf0 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.318+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2018fe20 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.318+0000 7f9d1e7fc700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d08038730 0x7f9d0803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.319+0000 7f9d1e7fc700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d08038730 0x7f9d0803abf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d14006fd0 tx=0x7f9d14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.323+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d10019cd0 con 0x7f9d201056d0 2024-05-08T21:48:24.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.524+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f9d20109d40 con 0x7f9d201056d0 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.525+0000 7f9d07fff700 1 -- 172.21.15.64:0/1357815448 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f9d10010b70 con 0x7f9d201056d0 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d08038730 msgr2=0x7f9d0803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d08038730 0x7f9d0803abf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d14006fd0 tx=0x7f9d14006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 msgr2=0x7f9d201907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d201907a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d10003e60 tx=0x7f9d10011a30 comp rx=0 tx=0).stop 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 shutdown_connections 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d08038730 0x7f9d0803abf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 --2- 172.21.15.64:0/1357815448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201056d0 0x7f9d201907a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 >> 172.21.15.64:0/1357815448 conn(0x7f9d20100cd0 msgr2=0x7f9d20101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 shutdown_connections 2024-05-08T21:48:24.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:24.527+0000 7f9d25470700 1 -- 172.21.15.64:0/1357815448 wait complete. 2024-05-08T21:48:24.906 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:48:24.906 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-08T21:48:24.913 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:48:24.914 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-08T21:48:24.914 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-08T21:48:24.914 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T21:48:24.915 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2024-05-08T21:48:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-05-08T21:48:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2024-05-08T21:48:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: Password: q1yo6innxh 2024-05-08T21:48:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:24.916 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config directory 2024-05-08T21:48:24.916 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling autotune for osd_memory_target 2024-05-08T21:48:25.857 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.281+0000 7fa17c9e0700 1 Processor -- start 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.281+0000 7fa17c9e0700 1 -- start start 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17c9e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa178109650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17c9e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa178074810 con 0x7fa178107260 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa178109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa178109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42400/0 (socket says 172.21.15.64:42400) 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17659c700 1 -- 172.21.15.64:0/452843168 learned_addr learned my addr 172.21.15.64:0/452843168 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17659c700 1 -- 172.21.15.64:0/452843168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178109b90 con 0x7fa178107260 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17659c700 1 --2- 172.21.15.64:0/452843168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa178109650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1680098a0 tx=0x7fa168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61808caff728257c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.282+0000 7fa17559a700 1 -- 172.21.15.64:0/452843168 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa168004160 con 0x7fa178107260 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17559a700 1 -- 172.21.15.64:0/452843168 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1680042c0 con 0x7fa178107260 2024-05-08T21:48:25.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17559a700 1 -- 172.21.15.64:0/452843168 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1680119b0 con 0x7fa178107260 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 -- 172.21.15.64:0/452843168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 msgr2=0x7fa178109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 --2- 172.21.15.64:0/452843168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa178109650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1680098a0 tx=0x7fa168009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 -- 172.21.15.64:0/452843168 shutdown_connections 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 --2- 172.21.15.64:0/452843168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa178109650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 -- 172.21.15.64:0/452843168 >> 172.21.15.64:0/452843168 conn(0x7fa178100c50 msgr2=0x7fa178103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 -- 172.21.15.64:0/452843168 shutdown_connections 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.283+0000 7fa17c9e0700 1 -- 172.21.15.64:0/452843168 wait complete. 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17c9e0700 1 Processor -- start 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17c9e0700 1 -- start start 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17c9e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa17819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17c9e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17819aac0 con 0x7fa178107260 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa17819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa17819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42414/0 (socket says 172.21.15.64:42414) 2024-05-08T21:48:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.284+0000 7fa17659c700 1 -- 172.21.15.64:0/605558294 learned_addr learned my addr 172.21.15.64:0/605558294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa17659c700 1 -- 172.21.15.64:0/605558294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009550 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa17659c700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa17819a580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa168011670 tx=0x7fa168011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa168003b20 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17819acc0 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa168003c80 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa168019770 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.285+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17819b160 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.286+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17810a970 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.286+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fa168019a30 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.286+0000 7fa1677fe700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa160038750 0x7fa16003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.286+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa16804caa0 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.287+0000 7fa175d9b700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa160038750 0x7fa16003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.287+0000 7fa175d9b700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa160038750 0x7fa16003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006fd0 tx=0x7fa16c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.291+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa168018800 con 0x7fa178107260 2024-05-08T21:48:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.491+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa17810abb0 con 0x7fa178107260 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.491+0000 7fa1677fe700 1 -- 172.21.15.64:0/605558294 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa168023e60 con 0x7fa178107260 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa160038750 msgr2=0x7fa16003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa160038750 0x7fa16003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006fd0 tx=0x7fa16c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 msgr2=0x7fa17819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa17819a580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa168011670 tx=0x7fa168011a30 comp rx=0 tx=0).stop 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 shutdown_connections 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa160038750 0x7fa16003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 --2- 172.21.15.64:0/605558294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178107260 0x7fa17819a580 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 >> 172.21.15.64:0/605558294 conn(0x7fa178100c50 msgr2=0x7fa178101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 shutdown_connections 2024-05-08T21:48:25.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:25.493+0000 7fa17c9e0700 1 -- 172.21.15.64:0/605558294 wait complete. 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e73d1b700 1 Processor -- start 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e73d1b700 1 -- start start 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e73d1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c105ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e73d1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1060a0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e71ab7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c105ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e71ab7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c105ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42418/0 (socket says 172.21.15.64:42418) 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.214+0000 7f6e71ab7700 1 -- 172.21.15.64:0/876546679 learned_addr learned my addr 172.21.15.64:0/876546679 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e71ab7700 1 -- 172.21.15.64:0/876546679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c1068b0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e71ab7700 1 --2- 172.21.15.64:0/876546679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c105ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e680098a0 tx=0x7f6e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38972bcab71c4707 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:26.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e70ab5700 1 -- 172.21.15.64:0/876546679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e68004160 con 0x7f6e6c1056b0 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e70ab5700 1 -- 172.21.15.64:0/876546679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e680042c0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e70ab5700 1 -- 172.21.15.64:0/876546679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e680119b0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e73d1b700 1 -- 172.21.15.64:0/876546679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 msgr2=0x7f6e6c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e73d1b700 1 --2- 172.21.15.64:0/876546679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c105ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e680098a0 tx=0x7f6e68009bb0 comp rx=0 tx=0).stop 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e73d1b700 1 -- 172.21.15.64:0/876546679 shutdown_connections 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.215+0000 7f6e73d1b700 1 --2- 172.21.15.64:0/876546679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c105ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 -- 172.21.15.64:0/876546679 >> 172.21.15.64:0/876546679 conn(0x7f6e6c100c50 msgr2=0x7f6e6c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 -- 172.21.15.64:0/876546679 shutdown_connections 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 -- 172.21.15.64:0/876546679 wait complete. 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 Processor -- start 2024-05-08T21:48:26.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 -- start start 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c1907d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e73d1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c190d10 con 0x7f6e6c1056b0 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e71ab7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c1907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e71ab7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c1907d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42434/0 (socket says 172.21.15.64:42434) 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.216+0000 7f6e71ab7700 1 -- 172.21.15.64:0/9306056 learned_addr learned my addr 172.21.15.64:0/9306056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e71ab7700 1 -- 172.21.15.64:0/9306056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009550 con 0x7f6e6c1056b0 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e71ab7700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c1907d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e68003e60 tx=0x7f6e68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e68003a60 con 0x7f6e6c1056b0 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e68003bc0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c18ef50 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e680196c0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.217+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c18f3f0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.218+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f6e68019930 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.218+0000 7f6e62ffd700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e58038820 0x7f6e5803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.218+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c0625d0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.218+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e6804cb00 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.218+0000 7f6e712b6700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e58038820 0x7f6e5803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.219+0000 7f6e712b6700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e58038820 0x7f6e5803ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.223+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e68019e60 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.542+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6e6c109ae0 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.544+0000 7f6e62ffd700 1 -- 172.21.15.64:0/9306056 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6e68010b70 con 0x7f6e6c1056b0 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e58038820 msgr2=0x7f6e5803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:26.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e58038820 0x7f6e5803ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 msgr2=0x7f6e6c1907d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c1907d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e68003e60 tx=0x7f6e68011a30 comp rx=0 tx=0).stop 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 shutdown_connections 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e58038820 0x7f6e5803ace0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 --2- 172.21.15.64:0/9306056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e6c1056b0 0x7f6e6c1907d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 >> 172.21.15.64:0/9306056 conn(0x7f6e6c100c50 msgr2=0x7f6e6c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.546+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 shutdown_connections 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-08T21:48:26.547+0000 7f6e73d1b700 1 -- 172.21.15.64:0/9306056 wait complete. 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T21:48:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:26.947 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2024-05-08T21:48:26.974 INFO:tasks.cephadm:Fetching config... 2024-05-08T21:48:26.974 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:48:26.975 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T21:48:26.993 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T21:48:26.993 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:48:26.993 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T21:48:27.052 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T21:48:27.052 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:48:27.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T21:48:27.121 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T21:48:27.122 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:48:27.122 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T21:48:27.181 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T21:48:27.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:48:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:27.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:48:27.320 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:27.333 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:48:27.379 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm6c09DJt/wV1GTaAbdN82mOL6Jdw/DEEs6qOhb3mVD70X4K9J+XHUnE7VdMthQu8MYcRTUzRTSK4Vwda0gPRhebS6MGsKDFrtRdS6SIcgKKUeHY57vmo3EUqemZor2CJsXVfpbL1ek7BeWmVqH00RhrgaYjr1oyJ7qXXocujL6h9yhYHB+PnZnVGhH8J8zp0pl1TCPlUZn5iLOyQhGMJET1ju4LuOvmFmGS+nsASkhP//IGfiuXDgengEfqUuxvZ4i3OcrYSStJl2T+Wr6XOYomxzzX0uIx+vFmUW46KuX+BdwvkGNQiMgedSeuYwaaKg8fnSDfdKqUVJ1TJXTy6ZlrQ5xVPLJesVxObseFAbUGgkVdEAZ8bxk23sr9KQaCFD4c4+2FEIZ8JTb/fKwFrraPF9qTHBp4u02bOdIzF/hgJVD1NwgFWfSjc6D01XFJOPSgfMH7RqXkut9K5LG93R0zmRy2SoBxCBi4o7r/yrG1Dr8VpBkUz5igEObI7v2rs= ceph-71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:27.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T21:48:27.416 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:27 smithi064 ceph-mon[42500]: mgrmap e13: x(active, since 6s) 2024-05-08T21:48:27.417 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/9306056' entity='client.admin' 2024-05-08T21:48:27.620 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3600648283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 msgr2=0x7fbdd80ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 --2- 172.21.15.64:0/3600648283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd80ffc50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009910 tx=0x7fbdc8009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3600648283 shutdown_connections 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 --2- 172.21.15.64:0/3600648283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd80ffc50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3600648283 >> 172.21.15.64:0/3600648283 conn(0x7fbdd80fadd0 msgr2=0x7fbdd80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3600648283 shutdown_connections 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.815+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3600648283 wait complete. 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbddc9f0700 1 Processor -- start 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbddc9f0700 1 -- start start 2024-05-08T21:48:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbddc9f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd818c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbddc9f0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd818cd20 con 0x7fbdd80ff830 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbdd659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd818c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbdd659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd818c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42450/0 (socket says 172.21.15.64:42450) 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbdd659c700 1 -- 172.21.15.64:0/3694514150 learned_addr learned my addr 172.21.15.64:0/3694514150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.816+0000 7fbdd659c700 1 -- 172.21.15.64:0/3694514150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80095f0 con 0x7fbdd80ff830 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.817+0000 7fbdd659c700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd818c7e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8005950 tx=0x7fbdc80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.817+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbdc801d070 con 0x7fbdd80ff830 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.817+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdc800bc10 con 0x7fbdd80ff830 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.817+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbdc8003a10 con 0x7fbdd80ff830 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.817+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd818af60 con 0x7fbdd80ff830 2024-05-08T21:48:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.817+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd818b400 con 0x7fbdd80ff830 2024-05-08T21:48:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.818+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fbdc8003c80 con 0x7fbdd80ff830 2024-05-08T21:48:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.818+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd80625d0 con 0x7fbdd80ff830 2024-05-08T21:48:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.818+0000 7fbdc77fe700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc00387f0 0x7fbdc003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.818+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbdc804c4a0 con 0x7fbdd80ff830 2024-05-08T21:48:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.818+0000 7fbdd5d9b700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc00387f0 0x7fbdc003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.818+0000 7fbdd5d9b700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc00387f0 0x7fbdc003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006fd0 tx=0x7fbdcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:28.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:28.823+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbdc8029950 con 0x7fbdd80ff830 2024-05-08T21:48:29.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.029+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fbdd8105580 con 0x7fbdd80ff830 2024-05-08T21:48:29.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.032+0000 7fbdc77fe700 1 -- 172.21.15.64:0/3694514150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbdc8026070 con 0x7fbdd80ff830 2024-05-08T21:48:29.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc00387f0 msgr2=0x7fbdc003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:29.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc00387f0 0x7fbdc003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006fd0 tx=0x7fbdcc006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 msgr2=0x7fbdd818c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd818c7e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8005950 tx=0x7fbdc80057c0 comp rx=0 tx=0).stop 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 shutdown_connections 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc00387f0 0x7fbdc003acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 --2- 172.21.15.64:0/3694514150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd80ff830 0x7fbdd818c7e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 >> 172.21.15.64:0/3694514150 conn(0x7fbdd80fadd0 msgr2=0x7fbdd80fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 shutdown_connections 2024-05-08T21:48:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:29.036+0000 7fbddc9f0700 1 -- 172.21.15.64:0/3694514150 wait complete. 2024-05-08T21:48:29.416 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T21:48:29.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T21:48:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:30.101 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:30 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3694514150' entity='client.admin' 2024-05-08T21:48:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 -- 172.21.15.64:0/411913636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 msgr2=0x7fe83c0ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 --2- 172.21.15.64:0/411913636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0ffc50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe830009910 tx=0x7fe830009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 -- 172.21.15.64:0/411913636 shutdown_connections 2024-05-08T21:48:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 --2- 172.21.15.64:0/411913636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0ffc50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 -- 172.21.15.64:0/411913636 >> 172.21.15.64:0/411913636 conn(0x7fe83c0fadd0 msgr2=0x7fe83c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 -- 172.21.15.64:0/411913636 shutdown_connections 2024-05-08T21:48:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 -- 172.21.15.64:0/411913636 wait complete. 2024-05-08T21:48:30.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.727+0000 7fe841fef700 1 Processor -- start 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe841fef700 1 -- start start 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe841fef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe841fef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c069910 con 0x7fe83c0ff830 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe83b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe83b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42456/0 (socket says 172.21.15.64:42456) 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe83b7fe700 1 -- 172.21.15.64:0/2293154189 learned_addr learned my addr 172.21.15.64:0/2293154189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe83b7fe700 1 -- 172.21.15.64:0/2293154189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300095f0 con 0x7fe83c0ff830 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.728+0000 7fe83b7fe700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0693d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe830005950 tx=0x7fe8300057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.729+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe83001d070 con 0x7fe83c0ff830 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.729+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe83000bc10 con 0x7fe83c0ff830 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.729+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe830003a10 con 0x7fe83c0ff830 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.729+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c198ae0 con 0x7fe83c0ff830 2024-05-08T21:48:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.729+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c198f80 con 0x7fe83c0ff830 2024-05-08T21:48:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.730+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fe830003cd0 con 0x7fe83c0ff830 2024-05-08T21:48:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.730+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c18e550 con 0x7fe83c0ff830 2024-05-08T21:48:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.730+0000 7fe838ff9700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe8240387a0 0x7fe82403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.730+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe83004c660 con 0x7fe83c0ff830 2024-05-08T21:48:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.730+0000 7fe83affd700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe8240387a0 0x7fe82403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.730+0000 7fe83affd700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe8240387a0 0x7fe82403ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe82c006fd0 tx=0x7fe82c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.735+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe83002aba0 con 0x7fe83c0ff830 2024-05-08T21:48:30.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.940+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe83c0613e0 con 0x7fe8240387a0 2024-05-08T21:48:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.943+0000 7fe838ff9700 1 -- 172.21.15.64:0/2293154189 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe83c0613e0 con 0x7fe8240387a0 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.945+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe8240387a0 msgr2=0x7fe82403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.945+0000 7fe841fef700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe8240387a0 0x7fe82403ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe82c006fd0 tx=0x7fe82c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.945+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 msgr2=0x7fe83c0693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.945+0000 7fe841fef700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0693d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe830005950 tx=0x7fe8300057c0 comp rx=0 tx=0).stop 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.946+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 shutdown_connections 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.946+0000 7fe841fef700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe8240387a0 0x7fe82403ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.946+0000 7fe841fef700 1 --2- 172.21.15.64:0/2293154189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83c0ff830 0x7fe83c0693d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.946+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 >> 172.21.15.64:0/2293154189 conn(0x7fe83c0fadd0 msgr2=0x7fe83c0fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.946+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 shutdown_connections 2024-05-08T21:48:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:30.946+0000 7fe841fef700 1 -- 172.21.15.64:0/2293154189 wait complete. 2024-05-08T21:48:31.342 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi160 2024-05-08T21:48:31.342 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:48:31.342 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.conf 2024-05-08T21:48:31.363 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:48:31.364 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:48:31.424 INFO:tasks.cephadm:Adding host smithi160 to orchestrator... 2024-05-08T21:48:31.424 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch host add smithi160 2024-05-08T21:48:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:31.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:48:31.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:31 smithi064 ceph-mon[42500]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-08T21:48:32.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:32 smithi064 ceph-mon[42500]: Updating smithi064:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:48:32.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:32 smithi064 ceph-mon[42500]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:48:32.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:32 smithi064 ceph-mon[42500]: Updating smithi064:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:48:32.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:32.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:32.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:32.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 -- 172.21.15.64:0/4007957021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 msgr2=0x7f9b24106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:32.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 --2- 172.21.15.64:0/4007957021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b24106ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009910 tx=0x7f9b0c009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:32.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 -- 172.21.15.64:0/4007957021 shutdown_connections 2024-05-08T21:48:32.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 --2- 172.21.15.64:0/4007957021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b24106ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 -- 172.21.15.64:0/4007957021 >> 172.21.15.64:0/4007957021 conn(0x7f9b240fc3d0 msgr2=0x7f9b240fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 -- 172.21.15.64:0/4007957021 shutdown_connections 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.589+0000 7f9b2881e700 1 -- 172.21.15.64:0/4007957021 wait complete. 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2881e700 1 Processor -- start 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2881e700 1 -- start start 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2881e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b241983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2881e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24198920 con 0x7f9b241068c0 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b241983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b241983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36532/0 (socket says 172.21.15.64:36532) 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2259c700 1 -- 172.21.15.64:0/1852141773 learned_addr learned my addr 172.21.15.64:0/1852141773 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.590+0000 7f9b2259c700 1 -- 172.21.15.64:0/1852141773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0095f0 con 0x7f9b241068c0 2024-05-08T21:48:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.591+0000 7f9b2259c700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b241983e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c006010 tx=0x7f9b0c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:32.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.591+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c01d070 con 0x7f9b241068c0 2024-05-08T21:48:32.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.591+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24198b80 con 0x7f9b241068c0 2024-05-08T21:48:32.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.591+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b0c00bbb0 con 0x7f9b241068c0 2024-05-08T21:48:32.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.591+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c003810 con 0x7f9b241068c0 2024-05-08T21:48:32.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.591+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b24199000 con 0x7f9b241068c0 2024-05-08T21:48:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.592+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f9b0c003a10 con 0x7f9b241068c0 2024-05-08T21:48:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.592+0000 7f9b1b7fe700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b100387a0 0x7f9b1003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.592+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9b0c04c440 con 0x7f9b241068c0 2024-05-08T21:48:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.592+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b24102000 con 0x7f9b241068c0 2024-05-08T21:48:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.592+0000 7f9b21d9b700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b100387a0 0x7f9b1003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.592+0000 7f9b21d9b700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b100387a0 0x7f9b1003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b14006fd0 tx=0x7f9b14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:32.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.597+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b0c026020 con 0x7f9b241068c0 2024-05-08T21:48:32.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:32.804+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b2410abc0 con 0x7f9b100387a0 2024-05-08T21:48:33.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:33 smithi064 ceph-mon[42500]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:34.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:34 smithi064 ceph-mon[42500]: Deploying cephadm binary to smithi160 2024-05-08T21:48:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.066+0000 7f9b1b7fe700 1 -- 172.21.15.64:0/1852141773 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9b2410abc0 con 0x7f9b100387a0 2024-05-08T21:48:35.066 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi160' with addr '172.21.15.160' 2024-05-08T21:48:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b100387a0 msgr2=0x7f9b1003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b100387a0 0x7f9b1003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b14006fd0 tx=0x7f9b14006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 msgr2=0x7f9b241983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b241983e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c006010 tx=0x7f9b0c004ff0 comp rx=0 tx=0).stop 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 shutdown_connections 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b100387a0 0x7f9b1003ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 --2- 172.21.15.64:0/1852141773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b241068c0 0x7f9b241983e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 >> 172.21.15.64:0/1852141773 conn(0x7f9b240fc3d0 msgr2=0x7f9b241084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 shutdown_connections 2024-05-08T21:48:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:35.068+0000 7f9b2881e700 1 -- 172.21.15.64:0/1852141773 wait complete. 2024-05-08T21:48:35.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T21:48:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:36.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:36.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:36 smithi064 ceph-mon[42500]: Added host smithi160 2024-05-08T21:48:36.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:36.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:36.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:36.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 -- 172.21.15.64:0/4160156243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 msgr2=0x7f2584106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:36.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 --2- 172.21.15.64:0/4160156243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f2584106ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2574009910 tx=0x7f2574009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:36.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 -- 172.21.15.64:0/4160156243 shutdown_connections 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 --2- 172.21.15.64:0/4160156243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f2584106ca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 -- 172.21.15.64:0/4160156243 >> 172.21.15.64:0/4160156243 conn(0x7f2584075350 msgr2=0x7f2584075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 -- 172.21.15.64:0/4160156243 shutdown_connections 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.539+0000 7f258a0d1700 1 -- 172.21.15.64:0/4160156243 wait complete. 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f258a0d1700 1 Processor -- start 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f258a0d1700 1 -- start start 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f258a0d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f25840ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f258a0d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584101760 con 0x7f25841068c0 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f25837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f25840ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f25837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f25840ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36542/0 (socket says 172.21.15.64:36542) 2024-05-08T21:48:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.540+0000 7f25837fe700 1 -- 172.21.15.64:0/85070657 learned_addr learned my addr 172.21.15.64:0/85070657 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f25837fe700 1 -- 172.21.15.64:0/85070657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740095f0 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f25837fe700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f25840ff8a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2574004f40 tx=0x7f2574005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f257401d070 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257400bbb0 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840ffde0 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2574003810 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.541+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2584100260 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.542+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f2574003a40 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.542+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584102000 con 0x7f25841068c0 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.542+0000 7f2580ff9700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f256c0387f0 0x7f256c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.542+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f257404c3e0 con 0x7f25841068c0 2024-05-08T21:48:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.542+0000 7f2582ffd700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f256c0387f0 0x7f256c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.543+0000 7f2582ffd700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f256c0387f0 0x7f256c03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2578006fd0 tx=0x7f2578006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:36.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.547+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2574029360 con 0x7f25841068c0 2024-05-08T21:48:36.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.755+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25841004a0 con 0x7f256c0387f0 2024-05-08T21:48:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.756+0000 7f2580ff9700 1 -- 172.21.15.64:0/85070657 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f25841004a0 con 0x7f256c0387f0 2024-05-08T21:48:36.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:36.757 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}] 2024-05-08T21:48:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f256c0387f0 msgr2=0x7f256c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f256c0387f0 0x7f256c03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2578006fd0 tx=0x7f2578006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 msgr2=0x7f25840ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f25840ff8a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2574004f40 tx=0x7f2574005020 comp rx=0 tx=0).stop 2024-05-08T21:48:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 shutdown_connections 2024-05-08T21:48:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f256c0387f0 0x7f256c03acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 --2- 172.21.15.64:0/85070657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25841068c0 0x7f25840ff8a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 >> 172.21.15.64:0/85070657 conn(0x7f2584075350 msgr2=0x7f25840fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 shutdown_connections 2024-05-08T21:48:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:36.759+0000 7f258a0d1700 1 -- 172.21.15.64:0/85070657 wait complete. 2024-05-08T21:48:37.129 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-05-08T21:48:37.130 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:48:37.130 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-05-08T21:48:37.150 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:48:37.151 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:48:37.211 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-05-08T21:48:37.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch host add smithi186 2024-05-08T21:48:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:37.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:37.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:37 smithi064 ceph-mon[42500]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:48:38.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 -- 172.21.15.64:0/1360492295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 msgr2=0x7eff74105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 --2- 172.21.15.64:0/1360492295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff74105c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff68009910 tx=0x7eff68009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 -- 172.21.15.64:0/1360492295 shutdown_connections 2024-05-08T21:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 --2- 172.21.15.64:0/1360492295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff74105c00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 -- 172.21.15.64:0/1360492295 >> 172.21.15.64:0/1360492295 conn(0x7eff74075350 msgr2=0x7eff74075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 -- 172.21.15.64:0/1360492295 shutdown_connections 2024-05-08T21:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.514+0000 7eff788cc700 1 -- 172.21.15.64:0/1360492295 wait complete. 2024-05-08T21:48:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff788cc700 1 Processor -- start 2024-05-08T21:48:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff788cc700 1 -- start start 2024-05-08T21:48:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff788cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff740fc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff788cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff740fe740 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff7259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff740fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff7259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff740fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36550/0 (socket says 172.21.15.64:36550) 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.515+0000 7eff7259c700 1 -- 172.21.15.64:0/706222939 learned_addr learned my addr 172.21.15.64:0/706222939 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff7259c700 1 -- 172.21.15.64:0/706222939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680095f0 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff7259c700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff740fc880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff68004f40 tx=0x7eff68005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff6801d070 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff68004370 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff740fcdc0 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff68003970 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.516+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff740fd1e0 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.517+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7eff68003c30 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.517+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff740ff330 con 0x7eff74105820 2024-05-08T21:48:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.517+0000 7eff637fe700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7eff5c0387a0 0x7eff5c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.517+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff6804c780 con 0x7eff74105820 2024-05-08T21:48:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.517+0000 7eff71d9b700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7eff5c0387a0 0x7eff5c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.518+0000 7eff71d9b700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7eff5c0387a0 0x7eff5c03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff64006fd0 tx=0x7eff64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:38.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.522+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7eff68029950 con 0x7eff74105820 2024-05-08T21:48:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:38.732+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) v1 -- 0x7eff740fd420 con 0x7eff5c0387a0 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:48:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:39 smithi064 ceph-mon[42500]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-08T21:48:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.009+0000 7eff637fe700 1 -- 172.21.15.64:0/706222939 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7eff740fd420 con 0x7eff5c0387a0 2024-05-08T21:48:41.010 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7eff5c0387a0 msgr2=0x7eff5c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7eff5c0387a0 0x7eff5c03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff64006fd0 tx=0x7eff64006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 msgr2=0x7eff740fc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff740fc880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff68004f40 tx=0x7eff68005020 comp rx=0 tx=0).stop 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 shutdown_connections 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7eff5c0387a0 0x7eff5c03ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 --2- 172.21.15.64:0/706222939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff74105820 0x7eff740fc880 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 >> 172.21.15.64:0/706222939 conn(0x7eff74075350 msgr2=0x7eff740680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 shutdown_connections 2024-05-08T21:48:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:41.011+0000 7eff788cc700 1 -- 172.21.15.64:0/706222939 wait complete. 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: Updating smithi160:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: Deploying cephadm binary to smithi186 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: Updating smithi160:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:41.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:41.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T21:48:41.619 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:42.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:41 smithi064 ceph-mon[42500]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:42.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:42.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:42.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:42.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 -- 172.21.15.64:0/3047771613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 msgr2=0x7fb5a4101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 --2- 172.21.15.64:0/3047771613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a4101e60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb594009910 tx=0x7fb594009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 -- 172.21.15.64:0/3047771613 shutdown_connections 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 --2- 172.21.15.64:0/3047771613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a4101e60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 -- 172.21.15.64:0/3047771613 >> 172.21.15.64:0/3047771613 conn(0x7fb5a40fb2b0 msgr2=0x7fb5a40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 -- 172.21.15.64:0/3047771613 shutdown_connections 2024-05-08T21:48:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.562+0000 7fb5a9a56700 1 -- 172.21.15.64:0/3047771613 wait complete. 2024-05-08T21:48:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a9a56700 1 Processor -- start 2024-05-08T21:48:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a9a56700 1 -- start start 2024-05-08T21:48:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a9a56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a41961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a9a56700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a4196710 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a41961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a41961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53884/0 (socket says 172.21.15.64:53884) 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a2ffd700 1 -- 172.21.15.64:0/2804790975 learned_addr learned my addr 172.21.15.64:0/2804790975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.563+0000 7fb5a2ffd700 1 -- 172.21.15.64:0/2804790975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940095f0 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.564+0000 7fb5a2ffd700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a41961d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb594004f40 tx=0x7fb594005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.564+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb59401d070 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.564+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb594004370 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.564+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a4196910 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.564+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb594003970 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.564+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a4196d30 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.565+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fb594003be0 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.565+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a40695e0 con 0x7fb5a4101a80 2024-05-08T21:48:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.565+0000 7fb5a8a54700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb58c0387f0 0x7fb58c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.565+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb59404c420 con 0x7fb5a4101a80 2024-05-08T21:48:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.565+0000 7fb5a27fc700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb58c0387f0 0x7fb58c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.565+0000 7fb5a27fc700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb58c0387f0 0x7fb58c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb598006fd0 tx=0x7fb598006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:42.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.570+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb594029950 con 0x7fb5a4101a80 2024-05-08T21:48:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.778+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5a410ac10 con 0x7fb58c0387f0 2024-05-08T21:48:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.779+0000 7fb5a8a54700 1 -- 172.21.15.64:0/2804790975 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fb5a410ac10 con 0x7fb58c0387f0 2024-05-08T21:48:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:48:42.779 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb58c0387f0 msgr2=0x7fb58c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb58c0387f0 0x7fb58c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb598006fd0 tx=0x7fb598006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 msgr2=0x7fb5a41961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a41961d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb594004f40 tx=0x7fb594005020 comp rx=0 tx=0).stop 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 shutdown_connections 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb58c0387f0 0x7fb58c03acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 --2- 172.21.15.64:0/2804790975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a4101a80 0x7fb5a41961d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 >> 172.21.15.64:0/2804790975 conn(0x7fb5a40fb2b0 msgr2=0x7fb5a4104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 shutdown_connections 2024-05-08T21:48:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:42.781+0000 7fb5a9a56700 1 -- 172.21.15.64:0/2804790975 wait complete. 2024-05-08T21:48:43.060 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:42 smithi064 ceph-mon[42500]: Added host smithi186 2024-05-08T21:48:43.061 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:43.154 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T21:48:43.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T21:48:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:48:44.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:43 smithi064 ceph-mon[42500]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:44.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:43 smithi064 ceph-mon[42500]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 -- 172.21.15.64:0/582670990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 msgr2=0x7fd82c105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 --2- 172.21.15.64:0/582670990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c105c00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd828009910 tx=0x7fd828009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 -- 172.21.15.64:0/582670990 shutdown_connections 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 --2- 172.21.15.64:0/582670990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c105c00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 -- 172.21.15.64:0/582670990 >> 172.21.15.64:0/582670990 conn(0x7fd82c075350 msgr2=0x7fd82c075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 -- 172.21.15.64:0/582670990 shutdown_connections 2024-05-08T21:48:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.321+0000 7fd83463a700 1 -- 172.21.15.64:0/582670990 wait complete. 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd83463a700 1 Processor -- start 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd83463a700 1 -- start start 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd83463a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c0fc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd83463a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c0fe740 con 0x7fd82c105820 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd8323d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c0fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd8323d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c0fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53900/0 (socket says 172.21.15.64:53900) 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.322+0000 7fd8323d6700 1 -- 172.21.15.64:0/1190080628 learned_addr learned my addr 172.21.15.64:0/1190080628 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd8323d6700 1 -- 172.21.15.64:0/1190080628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280095f0 con 0x7fd82c105820 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd8323d6700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c0fc880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd828004f40 tx=0x7fd828005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd82801d070 con 0x7fd82c105820 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c0fcdc0 con 0x7fd82c105820 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd828004370 con 0x7fd82c105820 2024-05-08T21:48:44.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd828003970 con 0x7fd82c105820 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.323+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c0fd1e0 con 0x7fd82c105820 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.324+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd828003c30 con 0x7fd82c105820 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.324+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c0ff330 con 0x7fd82c105820 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.324+0000 7fd81f7fe700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8180387a0 0x7fd81803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.324+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd82804c580 con 0x7fd82c105820 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.324+0000 7fd831bd5700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8180387a0 0x7fd81803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.324+0000 7fd831bd5700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8180387a0 0x7fd81803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fd0 tx=0x7fd820006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:44.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.329+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd82802aba0 con 0x7fd82c105820 2024-05-08T21:48:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.540+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd82c100ef0 con 0x7fd82c105820 2024-05-08T21:48:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.859+0000 7fd81f7fe700 1 -- 172.21.15.64:0/1190080628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd828026070 con 0x7fd82c105820 2024-05-08T21:48:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8180387a0 msgr2=0x7fd81803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8180387a0 0x7fd81803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fd0 tx=0x7fd820006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 msgr2=0x7fd82c0fc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c0fc880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd828004f40 tx=0x7fd828005020 comp rx=0 tx=0).stop 2024-05-08T21:48:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 shutdown_connections 2024-05-08T21:48:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8180387a0 0x7fd81803ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 --2- 172.21.15.64:0/1190080628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd82c105820 0x7fd82c0fc880 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 >> 172.21.15.64:0/1190080628 conn(0x7fd82c075350 msgr2=0x7fd82c0680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 shutdown_connections 2024-05-08T21:48:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:48:44.861+0000 7fd83463a700 1 -- 172.21.15.64:0/1190080628 wait complete. 2024-05-08T21:48:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-05-08T21:48:45.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1190080628' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T21:48:45.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:45.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:45.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:45.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:45.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:48:45.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:45.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:48:45.239 INFO:tasks.cephadm:Adding mon.a on smithi064 2024-05-08T21:48:45.239 INFO:tasks.cephadm:Adding mon.b on smithi160 2024-05-08T21:48:45.239 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-05-08T21:48:45.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c' 2024-05-08T21:48:45.498 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: Updating smithi186:/etc/ceph/ceph.conf 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1190080628' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: Updating smithi186:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:46.459 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 -- 172.21.15.186:0/567010297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 msgr2=0x7f0580106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:46.459 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 --2- 172.21.15.186:0/567010297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f0580106ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0570009910 tx=0x7f0570009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:46.459 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 -- 172.21.15.186:0/567010297 shutdown_connections 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 --2- 172.21.15.186:0/567010297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f0580106ce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 -- 172.21.15.186:0/567010297 >> 172.21.15.186:0/567010297 conn(0x7f05800fc410 msgr2=0x7f05800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 -- 172.21.15.186:0/567010297 shutdown_connections 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.458+0000 7f05880d6700 1 -- 172.21.15.186:0/567010297 wait complete. 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f05880d6700 1 Processor -- start 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f05880d6700 1 -- start start 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f05880d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f05800732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:46.460 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f05880d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0580075160 con 0x7f0580106900 2024-05-08T21:48:46.461 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f0585e72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f05800732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f0585e72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f05800732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.186:35436/0 (socket says 172.21.15.186:35436) 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f0585e72700 1 -- 172.21.15.186:0/969393685 learned_addr learned my addr 172.21.15.186:0/969393685 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.459+0000 7f0585e72700 1 -- 172.21.15.186:0/969393685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700095f0 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.460+0000 7f0585e72700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f05800732a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0570004f40 tx=0x7f0570005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.460+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f057001d070 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.460+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800737e0 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.460+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057000bbb0 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.460+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0570003810 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.460+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0580073c60 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.461+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f0570003a90 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.461+0000 7f0576ffd700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f056c0387a0 0x7f056c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.461+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0580104820 con 0x7f0580106900 2024-05-08T21:48:46.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.461+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f057004c430 con 0x7f0580106900 2024-05-08T21:48:46.463 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.461+0000 7f0585671700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f056c0387a0 0x7f056c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:46.463 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.462+0000 7f0585671700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f056c0387a0 0x7f056c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f057c006fd0 tx=0x7f057c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:46.467 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.466+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0570029770 con 0x7f0580106900 2024-05-08T21:48:46.668 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.667+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0580073ea0 con 0x7f056c0387a0 2024-05-08T21:48:46.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.674+0000 7f0576ffd700 1 -- 172.21.15.186:0/969393685 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0580073ea0 con 0x7f056c0387a0 2024-05-08T21:48:46.674 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f056c0387a0 msgr2=0x7f056c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f056c0387a0 0x7f056c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f057c006fd0 tx=0x7f057c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 msgr2=0x7f05800732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f05800732a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0570004f40 tx=0x7f0570005020 comp rx=0 tx=0).stop 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 shutdown_connections 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f056c0387a0 0x7f056c03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 --2- 172.21.15.186:0/969393685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580106900 0x7f05800732a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 >> 172.21.15.186:0/969393685 conn(0x7f05800fc410 msgr2=0x7f0580108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 shutdown_connections 2024-05-08T21:48:46.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:46.676+0000 7f05880d6700 1 -- 172.21.15.186:0/969393685 wait complete. 2024-05-08T21:48:47.062 DEBUG:teuthology.orchestra.run.smithi160:mon.b> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T21:48:47.064 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T21:48:47.066 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T21:48:47.066 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T21:48:47.096 INFO:journalctl@ceph.mon.c.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:48:47.100 INFO:journalctl@ceph.mon.b.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: Updating smithi186:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:48:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:47.632 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:48:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:47 smithi064 ceph-mon[42500]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:47 smithi064 ceph-mon[42500]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:47 smithi064 ceph-mon[42500]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c;count:3 2024-05-08T21:48:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:47 smithi064 ceph-mon[42500]: Deploying daemon mon.c on smithi186 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 -- 172.21.15.186:0/1441974354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 msgr2=0x7f3018105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 --2- 172.21.15.186:0/1441974354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018105cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3008009910 tx=0x7f3008009c20 comp rx=0 tx=0).stop 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 -- 172.21.15.186:0/1441974354 shutdown_connections 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 --2- 172.21.15.186:0/1441974354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018105cc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 -- 172.21.15.186:0/1441974354 >> 172.21.15.186:0/1441974354 conn(0x7f30180fa9f0 msgr2=0x7f30180fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 -- 172.21.15.186:0/1441974354 shutdown_connections 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.996+0000 7f301d50b700 1 -- 172.21.15.186:0/1441974354 wait complete. 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f301d50b700 1 Processor -- start 2024-05-08T21:48:48.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f301d50b700 1 -- start start 2024-05-08T21:48:49.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f301d50b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:49.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f301d50b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018198890 con 0x7f30181058e0 2024-05-08T21:48:49.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f3016ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:49.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f3016ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.186:35440/0 (socket says 172.21.15.186:35440) 2024-05-08T21:48:49.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.997+0000 7f3016ffd700 1 -- 172.21.15.186:0/3903079385 learned_addr learned my addr 172.21.15.186:0/3903079385 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:48:49.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f3016ffd700 1 -- 172.21.15.186:0/3903079385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080095f0 con 0x7f30181058e0 2024-05-08T21:48:49.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f3016ffd700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018198350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3008004f40 tx=0x7f3008005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f300801d070 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3018198af0 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f300800bbb0 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3008003810 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.998+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3018198f70 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.999+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3018102130 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.999+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f300800b6f0 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.999+0000 7f2ffffff700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30000387a0 0x7f300003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.999+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f300804c330 con 0x7f30181058e0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:48.999+0000 7f30167fc700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30000387a0 0x7f300003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:49.002 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.000+0000 7f30167fc700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30000387a0 0x7f300003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f300c006fd0 tx=0x7f300c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:49.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.004+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3008029360 con 0x7f30181058e0 2024-05-08T21:48:49.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.303+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f30181063d0 con 0x7f30181058e0 2024-05-08T21:48:49.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.304+0000 7f2ffffff700 1 -- 172.21.15.186:0/3903079385 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f30181063d0 con 0x7f30181058e0 2024-05-08T21:48:49.306 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:48:49.306 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","modified":"2024-05-08T21:47:26.900183Z","created":"2024-05-08T21:47:26.900183Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30000387a0 msgr2=0x7f300003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30000387a0 0x7f300003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f300c006fd0 tx=0x7f300c006e40 comp rx=0 tx=0).stop 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 msgr2=0x7f3018198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018198350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3008004f40 tx=0x7f3008005020 comp rx=0 tx=0).stop 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 shutdown_connections 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30000387a0 0x7f300003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 --2- 172.21.15.186:0/3903079385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30181058e0 0x7f3018198350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 >> 172.21.15.186:0/3903079385 conn(0x7f30180fa9f0 msgr2=0x7f3018108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 shutdown_connections 2024-05-08T21:48:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:49.306+0000 7f301d50b700 1 -- 172.21.15.186:0/3903079385 wait complete. 2024-05-08T21:48:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-05-08T21:48:50.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 systemd[1]: Started Ceph mon.c for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:48:50.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T21:48:50.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T21:48:50.142 INFO:journalctl@ceph.mon.c.smithi186.stdout: e1 2024-05-08T21:48:50.142 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:48:50.142 INFO:journalctl@ceph.mon.c.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout: legacy client fscid: -1 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout: No filesystems configured 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T21:48:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c;count:3 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: Deploying daemon mon.c on smithi186 2024-05-08T21:48:50.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:49 smithi186 ceph-mon[46619]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T21:48:50.720 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T21:48:50.721 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T21:48:50.949 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:48:54.923 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.920+0000 7f739dd06700 1 -- 172.21.15.186:0/3439930928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73800035c0 msgr2=0x7f7380005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:54.923 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.921+0000 7f739dd06700 1 --2- 172.21.15.186:0/3439930928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73800035c0 0x7f7380005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7398078170 tx=0x7f738800feb0 comp rx=0 tx=0).stop 2024-05-08T21:48:54.923 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.921+0000 7f739dd06700 1 -- 172.21.15.186:0/3439930928 shutdown_connections 2024-05-08T21:48:54.923 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.921+0000 7f739dd06700 1 --2- 172.21.15.186:0/3439930928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73800035c0 0x7f7380005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:54.924 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.921+0000 7f739dd06700 1 -- 172.21.15.186:0/3439930928 >> 172.21.15.186:0/3439930928 conn(0x7f7398076ea0 msgr2=0x7f73980772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:54.924 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.921+0000 7f739dd06700 1 -- 172.21.15.186:0/3439930928 shutdown_connections 2024-05-08T21:48:54.924 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.921+0000 7f739dd06700 1 -- 172.21.15.186:0/3439930928 wait complete. 2024-05-08T21:48:54.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.922+0000 7f739dd06700 1 Processor -- start 2024-05-08T21:48:54.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.922+0000 7f739dd06700 1 -- start start 2024-05-08T21:48:54.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.922+0000 7f739dd06700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:54.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.922+0000 7f739dd06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73981007b0 0x7f7398100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:54.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.922+0000 7f739dd06700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398102130 con 0x7f73981007b0 2024-05-08T21:48:54.930 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.922+0000 7f739dd06700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73981a2460 con 0x7f73800035c0 2024-05-08T21:48:54.930 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:54.930 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57354/0 (socket says 172.21.15.186:57354) 2024-05-08T21:48:54.930 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 -- 172.21.15.186:0/2486095352 learned_addr learned my addr 172.21.15.186:0/2486095352 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:48:54.930 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f7396ffd700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73981007b0 0x7f7398100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:54.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 -- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 msgr2=0x7f7398100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T21:48:54.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 -- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 msgr2=0x7f7398100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T21:48:54.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T21:48:54.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f73977fe700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:48:54.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f7396ffd700 1 -- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 msgr2=0x7f7398100270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:48:54.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f7396ffd700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:54.932 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f7396ffd700 1 -- 172.21.15.186:0/2486095352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738800f900 con 0x7f73981007b0 2024-05-08T21:48:54.932 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f7396ffd700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73981007b0 0x7f7398100c30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f738c00b4d0 tx=0x7f738c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:54.932 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.923+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f738c011630 con 0x7f73981007b0 2024-05-08T21:48:54.932 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.924+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f738c011c50 con 0x7f73981007b0 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.924+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73981a2600 con 0x7f73981007b0 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.924+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f738c010d90 con 0x7f73981007b0 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.924+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73981a2ad0 con 0x7f73981007b0 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.925+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f738c01a400 con 0x7f73981007b0 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.925+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7398068340 con 0x7f73981007b0 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.925+0000 7f7394ff9700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73840385e0 0x7f738403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:48:54.933 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.925+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f738c04ad90 con 0x7f73981007b0 2024-05-08T21:48:54.934 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.925+0000 7f73977fe700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73840385e0 0x7f738403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:48:54.934 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.926+0000 7f73977fe700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73840385e0 0x7f738403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f738800f800 tx=0x7f7388001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:48:54.934 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:54.930+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f738c050020 con 0x7f73981007b0 2024-05-08T21:48:55.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.207+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73980625d0 con 0x7f73981007b0 2024-05-08T21:48:55.210 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.208+0000 7f7394ff9700 1 -- 172.21.15.186:0/2486095352 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f738c0178b0 con 0x7f73981007b0 2024-05-08T21:48:55.211 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:48:55.211 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","modified":"2024-05-08T21:48:49.837794Z","created":"2024-05-08T21:47:26.900183Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T21:48:55.212 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73840385e0 msgr2=0x7f738403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73840385e0 0x7f738403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f738800f800 tx=0x7f7388001e50 comp rx=0 tx=0).stop 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73981007b0 msgr2=0x7f7398100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73981007b0 0x7f7398100c30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f738c00b4d0 tx=0x7f738c00b890 comp rx=0 tx=0).stop 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 shutdown_connections 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73840385e0 0x7f738403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73800035c0 0x7f7398100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 --2- 172.21.15.186:0/2486095352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73981007b0 0x7f7398100c30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 >> 172.21.15.186:0/2486095352 conn(0x7f7398076ea0 msgr2=0x7f73980fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 shutdown_connections 2024-05-08T21:48:55.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:48:55.211+0000 7f739dd06700 1 -- 172.21.15.186:0/2486095352 wait complete. 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: mon.a calling monitor election 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: mon.c calling monitor election 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: fsmap 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: mgrmap e13: x(active, since 34s) 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: overall HEALTH_OK 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:55.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:55.215 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: mon.a calling monitor election 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: mon.c calling monitor election 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:48:55.233 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:48:55.234 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: fsmap 2024-05-08T21:48:55.234 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:48:55.234 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: mgrmap e13: x(active, since 34s) 2024-05-08T21:48:55.234 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: overall HEALTH_OK 2024-05-08T21:48:55.234 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:55.234 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: Deploying daemon mon.b on smithi160 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/2486095352' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:48:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:48:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: Deploying daemon mon.b on smithi160 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/2486095352' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:48:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:48:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:56.568 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T21:48:56.568 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T21:48:56.794 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 systemd[1]: Started Ceph mon.b for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T21:48:57.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T21:48:57.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T21:48:57.800 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.table_properties_collectors: 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.inplace_update_support: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.bloom_locality: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.max_successive_merges: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T21:48:57.801 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.ttl: 2592000 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.enable_blob_files: false 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.min_blob_size: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T21:48:57.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 110620cf-c029-4c79-94e6-05b0d62ccf0d 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715204937488410, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715204937488859, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715204937, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "110620cf-c029-4c79-94e6-05b0d62ccf0d", "db_session_id": "FZ54Y00JDTR1811V1TN1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715204937488934, "job": 1, "event": "recovery_finished"} 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fd83762000 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: DB pointer 0x55fd83750000 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T21:48:57.803 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** DB Stats ** 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:48:57.804 INFO:journalctl@ceph.mon.b.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T21:48:57.805 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout: Block cache BinnedLRUCache@0x55fd836c51f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: using public_addr v2:172.21.15.160:0/0 -> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: starting mon.b rank -1 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(???) e0 preinit fsid 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:48:57.806 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout: e1 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout: legacy client fscid: -1 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout: No filesystems configured 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T21:48:57.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c;count:3 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: Deploying daemon mon.c on smithi186 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.a calling monitor election 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.c calling monitor election 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: fsmap 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mgrmap e13: x(active, since 34s) 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: overall HEALTH_OK 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:48:57.809 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:48:57.810 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:48:57.810 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: Deploying daemon mon.b on smithi160 2024-05-08T21:48:57.810 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/2486095352' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:48:57.810 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:48:57.810 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:48:57 smithi160 ceph-mon[46962]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T21:49:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:49:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: mon.c calling monitor election 2024-05-08T21:49:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: mon.a calling monitor election 2024-05-08T21:49:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:49:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: fsmap 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: mgrmap e13: x(active, since 42s) 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T21:49:02.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: mon.b (rank 2) addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] is down (out of quorum) 2024-05-08T21:49:02.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: mon.c calling monitor election 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: mon.a calling monitor election 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: fsmap 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: mgrmap e13: x(active, since 42s) 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: mon.b (rank 2) addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] is down (out of quorum) 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:02.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.769 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 -- 172.21.15.186:0/1810010717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 msgr2=0x7f7d94008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:03.769 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 --2- 172.21.15.186:0/1810010717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7d94008710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7da4101aa0 tx=0x7f7d9800fe90 comp rx=0 tx=0).stop 2024-05-08T21:49:03.769 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 -- 172.21.15.186:0/1810010717 shutdown_connections 2024-05-08T21:49:03.770 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 --2- 172.21.15.186:0/1810010717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7d94008710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:03.770 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 -- 172.21.15.186:0/1810010717 >> 172.21.15.186:0/1810010717 conn(0x7f7da40fc4f0 msgr2=0x7f7da40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:03.770 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 -- 172.21.15.186:0/1810010717 shutdown_connections 2024-05-08T21:49:03.770 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.769+0000 7f7daad5d700 1 -- 172.21.15.186:0/1810010717 wait complete. 2024-05-08T21:49:03.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 Processor -- start 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 -- start start 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7da4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 0x7f7da419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da41970a0 con 0x7f7d940062a0 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da41a3010 con 0x7f7da41969b0 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7daad5d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da41a3150 con 0x7f7da419b2a0 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da8af9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7da4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:38268/0 (socket says 172.21.15.186:38268) 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 -- 172.21.15.186:0/399230562 learned_addr learned my addr 172.21.15.186:0/399230562 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 msgr2=0x7f7da419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T21:49:03.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 msgr2=0x7f7da419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da3fff700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 0x7f7da419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da92fa700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da3fff700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 msgr2=0x7f7da419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da3fff700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 msgr2=0x7f7da419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da3fff700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 0x7f7da419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da3fff700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 0x7f7da419ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da8af9700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 msgr2=0x7f7da419b750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da8af9700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da8af9700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 msgr2=0x7f7da419ad60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.770+0000 7f7da8af9700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 0x7f7da419ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7da8af9700 1 -- 172.21.15.186:0/399230562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9800f8e0 con 0x7f7d940062a0 2024-05-08T21:49:03.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7da8af9700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7da4196470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d98010010 tx=0x7f7d9801dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9801dca0 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d98005460 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da41a33d0 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98005da0 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.771+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da41a3920 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.772+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f7d98011880 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.772+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da41900b0 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.772+0000 7f7da1ffb700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d88038c70 0x7f7d8803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.772+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d9804e230 con 0x7f7d940062a0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.772+0000 7f7da3fff700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d88038c70 0x7f7d8803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:03.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.773+0000 7f7da3fff700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d88038c70 0x7f7d8803b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009850 tx=0x7f7d90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:03.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:03.778+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d98024310 con 0x7f7d940062a0 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:03.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.089 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.088+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7da4102120 con 0x7f7d940062a0 2024-05-08T21:49:04.089 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.089+0000 7f7da1ffb700 1 -- 172.21.15.186:0/399230562 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f7d98038b00 con 0x7f7d940062a0 2024-05-08T21:49:04.090 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-08T21:49:04.090 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","modified":"2024-05-08T21:48:57.519738Z","created":"2024-05-08T21:47:26.900183Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d88038c70 msgr2=0x7f7d8803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d88038c70 0x7f7d8803b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009850 tx=0x7f7d90008040 comp rx=0 tx=0).stop 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 msgr2=0x7f7da4196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7da4196470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d98010010 tx=0x7f7d9801dae0 comp rx=0 tx=0).stop 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 shutdown_connections 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d88038c70 0x7f7d8803b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:04.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da419b2a0 0x7f7da419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:04.093 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d940062a0 0x7f7da4196470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:04.093 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 --2- 172.21.15.186:0/399230562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7da41969b0 0x7f7da419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:04.093 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.091+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 >> 172.21.15.186:0/399230562 conn(0x7f7da40fc4f0 msgr2=0x7f7da4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:04.093 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.092+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 shutdown_connections 2024-05-08T21:49:04.093 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:04.092+0000 7f7daad5d700 1 -- 172.21.15.186:0/399230562 wait complete. 2024-05-08T21:49:04.093 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-05-08T21:49:04.503 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T21:49:04.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: mon.c calling monitor election 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: mon.a calling monitor election 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: fsmap 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: mgrmap e13: x(active, since 42s) 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: mon.b (rank 2) addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] is down (out of quorum) 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:04.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:04.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:05.036 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: mon.b calling monitor election 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: Reconfiguring daemon mon.a on smithi064 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: mon.b calling monitor election 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: mon.c calling monitor election 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: mon.a calling monitor election 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T21:49:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: fsmap 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: mgrmap e13: x(active, since 44s) 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: Cluster is now healthy 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: overall HEALTH_OK 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:05.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: mon.b calling monitor election 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: Reconfiguring daemon mon.a on smithi064 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: mon.b calling monitor election 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: mon.c calling monitor election 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: mon.a calling monitor election 2024-05-08T21:49:05.492 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: fsmap 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: mgrmap e13: x(active, since 44s) 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: Cluster is now healthy 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: overall HEALTH_OK 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:49:05.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:05.494 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: mon.b calling monitor election 2024-05-08T21:49:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T21:49:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: Reconfiguring daemon mon.a on smithi064 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: mon.b calling monitor election 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: mon.c calling monitor election 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: mon.a calling monitor election 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: fsmap 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: mgrmap e13: x(active, since 44s) 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: Cluster is now healthy 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: overall HEALTH_OK 2024-05-08T21:49:05.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:49:05.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:49:05.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:05.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 -- 172.21.15.64:0/606475544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f800ffed0 msgr2=0x7f9f80100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 --2- 172.21.15.64:0/606475544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f800ffed0 0x7f9f80100350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009990 tx=0x7f9f74009ca0 comp rx=0 tx=0).stop 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 -- 172.21.15.64:0/606475544 shutdown_connections 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 --2- 172.21.15.64:0/606475544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f80100890 0x7f9f8010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 --2- 172.21.15.64:0/606475544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f80105f00 0x7f9f801062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 --2- 172.21.15.64:0/606475544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f800ffed0 0x7f9f80100350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 -- 172.21.15.64:0/606475544 >> 172.21.15.64:0/606475544 conn(0x7f9f80075f80 msgr2=0x7f9f80076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 -- 172.21.15.64:0/606475544 shutdown_connections 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 -- 172.21.15.64:0/606475544 wait complete. 2024-05-08T21:49:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.990+0000 7f9f85d76700 1 Processor -- start 2024-05-08T21:49:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 -- start start 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f800ffed0 0x7f9f80197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f80100890 0x7f9f80197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 0x7f9f8019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019c320 con 0x7f9f80105f00 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019c490 con 0x7f9f80100890 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f85d76700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019c650 con 0x7f9f800ffed0 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 0x7f9f8019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 0x7f9f8019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51920/0 (socket says 172.21.15.64:51920) 2024-05-08T21:49:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 -- 172.21.15.64:0/2205637835 learned_addr learned my addr 172.21.15.64:0/2205637835 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7effd700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f80100890 0x7f9f80197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7f7fe700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f800ffed0 0x7f9f80197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 -- 172.21.15.64:0/2205637835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f800ffed0 msgr2=0x7f9f80197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f800ffed0 0x7f9f80197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 -- 172.21.15.64:0/2205637835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f80100890 msgr2=0x7f9f80197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f80100890 0x7f9f80197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 -- 172.21.15.64:0/2205637835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740095f0 con 0x7f9f80105f00 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.991+0000 7f9f7ffff700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 0x7f9f8019bd10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f78007d70 tx=0x7f9f7800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.992+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800ad40 con 0x7f9f80105f00 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.992+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8019c930 con 0x7f9f80105f00 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.992+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f780128e0 con 0x7f9f80105f00 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.992+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f780116b0 con 0x7f9f80105f00 2024-05-08T21:49:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.992+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f801a4190 con 0x7f9f80105f00 2024-05-08T21:49:05.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.993+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f9f78011870 con 0x7f9f80105f00 2024-05-08T21:49:05.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.993+0000 7f9f7cff9700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9f680389d0 0x7f9f6803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:05.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.993+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9f7804b500 con 0x7f9f80105f00 2024-05-08T21:49:05.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.993+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f801015b0 con 0x7f9f80105f00 2024-05-08T21:49:05.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.993+0000 7f9f7f7fe700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9f680389d0 0x7f9f6803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:05.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.993+0000 7f9f7f7fe700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9f680389d0 0x7f9f6803ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f70006fd0 tx=0x7f9f70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:05.998+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f78010cf0 con 0x7f9f80105f00 2024-05-08T21:49:06.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.217+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9f80198730 con 0x7f9f80105f00 2024-05-08T21:49:06.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.217+0000 7f9f7cff9700 1 -- 172.21.15.64:0/2205637835 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f9f7803a090 con 0x7f9f80105f00 2024-05-08T21:49:06.219 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:49:06.219 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-05-08T21:49:06.219 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:49:06.219 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-05-08T21:49:06.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9f680389d0 msgr2=0x7f9f6803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9f680389d0 0x7f9f6803ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f70006fd0 tx=0x7f9f70008040 comp rx=0 tx=0).stop 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 msgr2=0x7f9f8019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 0x7f9f8019bd10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f78007d70 tx=0x7f9f7800a340 comp rx=0 tx=0).stop 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 shutdown_connections 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9f680389d0 0x7f9f6803ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f80105f00 0x7f9f8019bd10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f800ffed0 0x7f9f80197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 --2- 172.21.15.64:0/2205637835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f80100890 0x7f9f80197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 >> 172.21.15.64:0/2205637835 conn(0x7f9f80075f80 msgr2=0x7f9f800ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 shutdown_connections 2024-05-08T21:49:06.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:06.219+0000 7f9f85d76700 1 -- 172.21.15.64:0/2205637835 wait complete. 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: Reconfiguring mon.b (monmap changed)... 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: Reconfiguring daemon mon.b on smithi160 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:49:06.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:06.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: Reconfiguring mon.b (monmap changed)... 2024-05-08T21:49:06.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: Reconfiguring daemon mon.b on smithi160 2024-05-08T21:49:06.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:06.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:06.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:06.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:49:06.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:49:06.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: Reconfiguring mon.b (monmap changed)... 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: Reconfiguring daemon mon.b on smithi160 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:49:06.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:06.583 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T21:49:06.584 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:49:06.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T21:49:06.614 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:49:06.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:49:06.682 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:49:06.682 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T21:49:06.718 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:49:06.718 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:49:06.789 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:49:06.789 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T21:49:06.821 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:49:06.821 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:49:06.888 INFO:tasks.cephadm:Adding mgr.x on smithi064 2024-05-08T21:49:06.888 INFO:tasks.cephadm:Adding mgr.y on smithi160 2024-05-08T21:49:06.888 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-05-08T21:49:06.889 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi064=x;smithi160=y;smithi186=z' 2024-05-08T21:49:07.144 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: Reconfiguring mon.c (monmap changed)... 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: Reconfiguring daemon mon.c on smithi186 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2205637835' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:07.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:07.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: Reconfiguring mon.c (monmap changed)... 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: Reconfiguring daemon mon.c on smithi186 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2205637835' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:07.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: Reconfiguring mon.c (monmap changed)... 2024-05-08T21:49:07.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: Reconfiguring daemon mon.c on smithi186 2024-05-08T21:49:07.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2205637835' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:07.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:07.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:07.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:07.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:07.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:08.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 -- 172.21.15.186:0/1153927849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180102b80 msgr2=0x7fa180109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 --2- 172.21.15.186:0/1153927849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180102b80 0x7fa180109430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa17800b0a0 tx=0x7fa17800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 -- 172.21.15.186:0/1153927849 shutdown_connections 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 --2- 172.21.15.186:0/1153927849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180102b80 0x7fa180109430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 --2- 172.21.15.186:0/1153927849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180100fe0 0x7fa180101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 --2- 172.21.15.186:0/1153927849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 0x7fa180102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 -- 172.21.15.186:0/1153927849 >> 172.21.15.186:0/1153927849 conn(0x7fa1800fc580 msgr2=0x7fa1800fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 -- 172.21.15.186:0/1153927849 shutdown_connections 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.086+0000 7fa18532c700 1 -- 172.21.15.186:0/1153927849 wait complete. 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.087+0000 7fa18532c700 1 Processor -- start 2024-05-08T21:49:08.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.087+0000 7fa18532c700 1 -- start start 2024-05-08T21:49:08.090 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 0x7fa180196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:08.090 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 0x7fa1801974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180102b80 0x7fa18019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019eac0 con 0x7fa180102b80 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019ec00 con 0x7fa1801021e0 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019ed90 con 0x7fa180100fe0 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 0x7fa180196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 0x7fa180196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:38288/0 (socket says 172.21.15.186:38288) 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 -- 172.21.15.186:0/3339320206 learned_addr learned my addr 172.21.15.186:0/3339320206 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17e7fc700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 0x7fa1801974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17f7fe700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180102b80 0x7fa18019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 -- 172.21.15.186:0/3339320206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 msgr2=0x7fa1801974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 0x7fa1801974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 -- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180102b80 msgr2=0x7fa18019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180102b80 0x7fa18019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 -- 172.21.15.186:0/3339320206 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178009d00 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17e7fc700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 0x7fa1801974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17f7fe700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180102b80 0x7fa18019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa17effd700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 0x7fa180196f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c9a0 tx=0x7fa17400cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174004220 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa174007d20 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18019f070 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174003d00 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.088+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18019f570 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.089+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa17400f450 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.089+0000 7fa167fff700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680389d0 0x7fa16803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.089+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa17404b0b0 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.089+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1801083c0 con 0x7fa180100fe0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.089+0000 7fa17e7fc700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680389d0 0x7fa16803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:08.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.090+0000 7fa17e7fc700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680389d0 0x7fa16803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa170005950 tx=0x7fa170005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:08.095 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.095+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa1740173b0 con 0x7fa180100fe0 2024-05-08T21:49:08.297 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.297+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi160=y;smithi186=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1800613e0 con 0x7fa1680389d0 2024-05-08T21:49:08.303 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.303+0000 7fa167fff700 1 -- 172.21.15.186:0/3339320206 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa1800613e0 con 0x7fa1680389d0 2024-05-08T21:49:08.304 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-05-08T21:49:08.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.305+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680389d0 msgr2=0x7fa16803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:08.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.305+0000 7fa18532c700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680389d0 0x7fa16803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa170005950 tx=0x7fa170005f50 comp rx=0 tx=0).stop 2024-05-08T21:49:08.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.305+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 msgr2=0x7fa180196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:08.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.305+0000 7fa18532c700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 0x7fa180196f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c9a0 tx=0x7fa17400cd60 comp rx=0 tx=0).stop 2024-05-08T21:49:08.306 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 shutdown_connections 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680389d0 0x7fa16803ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa180102b80 0x7fa18019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180100fe0 0x7fa180196f80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 --2- 172.21.15.186:0/3339320206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1801021e0 0x7fa1801974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 >> 172.21.15.186:0/3339320206 conn(0x7fa1800fc580 msgr2=0x7fa180105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 shutdown_connections 2024-05-08T21:49:08.307 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:08.306+0000 7fa18532c700 1 -- 172.21.15.186:0/3339320206 wait complete. 2024-05-08T21:49:08.697 DEBUG:teuthology.orchestra.run.smithi160:mgr.y> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T21:49:08.699 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T21:49:08.701 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T21:49:08.702 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:49:08.702 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T21:49:08.723 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T21:49:08.723 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-05-08T21:49:08.731 INFO:journalctl@ceph.mgr.z.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:49:08.731 INFO:journalctl@ceph.mgr.y.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 185415 Links: 1 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-08 21:48:29.435496064 +0000 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-08 21:41:58.785265259 +0000 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-08 21:41:58.785265259 +0000 2024-05-08T21:49:08.785 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-05-08T21:49:08.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T21:49:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-08T21:49:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-08T21:49:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00024049 s, 2.1 MB/s 2024-05-08T21:49:08.859 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T21:49:08.920 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 179748 Links: 1 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-08 21:48:29.485495104 +0000 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-08 21:41:59.124258648 +0000 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-08 21:41:59.124258648 +0000 2024-05-08T21:49:08.980 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-05-08T21:49:08.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T21:49:09.049 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-08T21:49:09.049 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-08T21:49:09.049 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000221017 s, 2.3 MB/s 2024-05-08T21:49:09.050 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T21:49:09.110 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 183552 Links: 1 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-08 21:48:29.535494143 +0000 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-08 21:41:59.477251764 +0000 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-08 21:41:59.477251764 +0000 2024-05-08T21:49:09.169 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-05-08T21:49:09.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T21:49:09.236 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-08T21:49:09.237 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-08T21:49:09.237 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000187799 s, 2.7 MB/s 2024-05-08T21:49:09.238 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T21:49:09.297 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-05-08T21:49:09.355 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T21:49:09.355 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:09.355 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 182707 Links: 1 2024-05-08T21:49:09.355 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:09.355 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:09.355 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-08 21:48:29.582493240 +0000 2024-05-08T21:49:09.356 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-08 21:41:59.815245172 +0000 2024-05-08T21:49:09.356 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-08 21:41:59.815245172 +0000 2024-05-08T21:49:09.356 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-05-08T21:49:09.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T21:49:09.421 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-08T21:49:09.421 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-08T21:49:09.421 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00023323 s, 2.2 MB/s 2024-05-08T21:49:09.423 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T21:49:09.482 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T21:49:09.482 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T21:49:09.503 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T21:49:09.503 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 185346 Links: 1 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 21:48:57.124100086 +0000 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 21:41:45.025175791 +0000 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 21:41:45.025175791 +0000 2024-05-08T21:49:09.565 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T21:49:09.566 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi160=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: Saving service mgr spec with placement smithi064=x;smithi160=y;smithi186=z;count:3 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:09.611 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:49:09.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:09.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:09.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mon[46619]: Deploying daemon mgr.z on smithi186 2024-05-08T21:49:09.632 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:09.632 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi160=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:09.632 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: Saving service mgr spec with placement smithi064=x;smithi160=y;smithi186=z;count:3 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:09 smithi160 ceph-mon[46962]: Deploying daemon mgr.z on smithi186 2024-05-08T21:49:09.635 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T21:49:09.635 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T21:49:09.635 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000285518 s, 1.8 MB/s 2024-05-08T21:49:09.637 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T21:49:09.698 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi160=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: Saving service mgr spec with placement smithi064=x;smithi160=y;smithi186=z;count:3 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:09.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:09 smithi064 ceph-mon[42500]: Deploying daemon mgr.z on smithi186 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 185423 Links: 1 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 21:48:57.124100086 +0000 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 21:41:45.375170799 +0000 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 21:41:45.375170799 +0000 2024-05-08T21:49:09.758 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T21:49:09.758 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T21:49:09.828 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T21:49:09.828 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T21:49:09.828 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000273873 s, 1.9 MB/s 2024-05-08T21:49:09.829 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T21:49:09.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:09 smithi186 systemd[1]: Starting Ceph mgr.z for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:49:09.890 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-05-08T21:49:09.950 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 185531 Links: 1 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 21:48:57.124100086 +0000 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 21:41:45.715165951 +0000 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 21:41:45.715165951 +0000 2024-05-08T21:49:09.951 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T21:49:09.951 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T21:49:10.020 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T21:49:10.020 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T21:49:10.020 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000209128 s, 2.4 MB/s 2024-05-08T21:49:10.021 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T21:49:10.082 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-05-08T21:49:10.140 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 182482 Links: 1 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 21:48:57.124100086 +0000 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 21:41:46.256158237 +0000 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 21:41:46.256158237 +0000 2024-05-08T21:49:10.141 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T21:49:10.141 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T21:49:10.208 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T21:49:10.208 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T21:49:10.209 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000215177 s, 2.4 MB/s 2024-05-08T21:49:10.209 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T21:49:10.267 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-08T21:49:10.267 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T21:49:10.285 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T21:49:10.285 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-05-08T21:49:10.343 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 175985 Links: 1 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-08 21:49:09.560623034 +0000 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-08 21:41:53.658653298 +0000 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-08 21:41:53.658653298 +0000 2024-05-08T21:49:10.344 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2024-05-08T21:49:10.344 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T21:49:10.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:09 smithi186 ceph-mgr[48504]: -- 172.21.15.186:0/194864458 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634a0ce21a0 con 0x5634a0cb8800 2024-05-08T21:49:10.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:10 smithi186 bash[48466]: cbe67a60fa0908c5df51954644c5c4d3968cb4e8348554d4e9fff35afaf3bfe2 2024-05-08T21:49:10.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:10 smithi186 systemd[1]: Started Ceph mgr.z for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:49:10.411 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:10 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:10.393+0000 7f24d6d11200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:49:10.414 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-08T21:49:10.414 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-08T21:49:10.414 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000216312 s, 2.4 MB/s 2024-05-08T21:49:10.415 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T21:49:10.475 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-05-08T21:49:10.534 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T21:49:10.534 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 176053 Links: 1 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-08 21:49:09.560623034 +0000 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-08 21:41:53.999647161 +0000 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-08 21:41:53.999647161 +0000 2024-05-08T21:49:10.535 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2024-05-08T21:49:10.535 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T21:49:10.602 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-08T21:49:10.602 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-08T21:49:10.602 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000196734 s, 2.6 MB/s 2024-05-08T21:49:10.603 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T21:49:10.663 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-05-08T21:49:10.680 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:10 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:10.494+0000 7f24d6d11200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:49:10.680 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:10 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:10.589+0000 7f24d6d11200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:49:10.681 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:10 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:10.680+0000 7f24d6d11200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 180953 Links: 1 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-08 21:49:09.560623034 +0000 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-08 21:41:54.356640736 +0000 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-08 21:41:54.356640736 +0000 2024-05-08T21:49:10.699 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2024-05-08T21:49:10.700 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T21:49:10.767 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-08T21:49:10.767 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-08T21:49:10.767 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000187027 s, 2.7 MB/s 2024-05-08T21:49:10.768 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T21:49:10.826 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 178800 Links: 1 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-08 21:49:09.560623034 +0000 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-08 21:41:54.700634546 +0000 2024-05-08T21:49:10.884 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-08 21:41:54.700634546 +0000 2024-05-08T21:49:10.885 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2024-05-08T21:49:10.885 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T21:49:10.950 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-08T21:49:10.950 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-08T21:49:10.950 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000244445 s, 2.1 MB/s 2024-05-08T21:49:10.951 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T21:49:11.009 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2024-05-08T21:49:11.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T21:49:11.247 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:49:11.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:11.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:11.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:49:11.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:11.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:11.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mon[46962]: Deploying daemon mgr.y on smithi160 2024-05-08T21:49:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:11 smithi186 ceph-mon[46619]: Deploying daemon mgr.y on smithi160 2024-05-08T21:49:11.360 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:11.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:11 smithi064 ceph-mon[42500]: Deploying daemon mgr.y on smithi160 2024-05-08T21:49:11.544 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:11 smithi160 systemd[1]: Starting Ceph mgr.y for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:49:11.593 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:11.342+0000 7f24d6d11200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:49:11.891 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:11.593+0000 7f24d6d11200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:49:11.891 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:11.680+0000 7f24d6d11200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:49:11.891 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:11.790+0000 7f24d6d11200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:49:12.200 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:11 smithi160 ceph-mgr[48470]: -- 172.21.15.160:0/1537909430 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a7a2c01a0 con 0x564a7a296c00 2024-05-08T21:49:12.200 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:11 smithi160 bash[48432]: b738fb96d6bb7cfb49bab2034874a5c92333337448dcaea9e98e971492e06b52 2024-05-08T21:49:12.200 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:11 smithi160 systemd[1]: Started Ceph mgr.y for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:49:12.295 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:12.198+0000 7f98f0a9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:49:12.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:11.951+0000 7f24d6d11200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:49:12.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:12.081+0000 7f24d6d11200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:49:12.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:12.171+0000 7f24d6d11200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:49:12.494 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:12.293+0000 7f98f0a9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:49:12.494 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:12.392+0000 7f98f0a9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:49:12.494 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:12.493+0000 7f98f0a9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:49:12.676 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T21:49:12.677 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T21:49:12.677 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-08T21:49:12.677 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-08T21:49:12.677 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114831 s, 913 MB/s 2024-05-08T21:49:12.677 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-08T21:49:12.869 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:12.594+0000 7f24d6d11200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:49:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:13.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-05-08T21:49:13.141 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:13.010+0000 7f24d6d11200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:49:13.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:13.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:13.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:13.315 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:49:13.416 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.164+0000 7f98f0a9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:49:13.641 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:13.208+0000 7f24d6d11200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:49:13.641 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:13.301+0000 7f24d6d11200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:49:13.767 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.415+0000 7f98f0a9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:49:13.768 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.500+0000 7f98f0a9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:49:13.768 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.608+0000 7f98f0a9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:49:13.768 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.766+0000 7f98f0a9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:49:14.044 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.894+0000 7f98f0a9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:13.978+0000 7f98f0a9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:14.068+0000 7f24d6d11200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:14.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:14.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:49:14.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.493+0000 7f6c5320d700 1 -- 172.21.15.64:0/1393288935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c0fff00 msgr2=0x7f6c4c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.493+0000 7f6c5320d700 1 --2- 172.21.15.64:0/1393288935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c100320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009990 tx=0x7f6c40009ca0 comp rx=0 tx=0).stop 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- 172.21.15.64:0/1393288935 shutdown_connections 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 --2- 172.21.15.64:0/1393288935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c101a20 0x7f6c4c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 --2- 172.21.15.64:0/1393288935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c100320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 --2- 172.21.15.64:0/1393288935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c101060 0x7f6c4c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- 172.21.15.64:0/1393288935 >> 172.21.15.64:0/1393288935 conn(0x7f6c4c0fb4a0 msgr2=0x7f6c4c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- 172.21.15.64:0/1393288935 shutdown_connections 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- 172.21.15.64:0/1393288935 wait complete. 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 Processor -- start 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- start start 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c103de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c101060 0x7f6c4c1024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 0x7f6c4c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1030f0 con 0x7f6c4c101a20 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c103260 con 0x7f6c4c0fff00 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.494+0000 7f6c5320d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1a1020 con 0x7f6c4c101060 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 0x7f6c4c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c50fa9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 0x7f6c4c102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33136/0 (socket says 172.21.15.64:33136) 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 -- 172.21.15.64:0/2282721854 learned_addr learned my addr 172.21.15.64:0/2282721854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c4bfff700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c101060 0x7f6c4c1024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 -- 172.21.15.64:0/2282721854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c101060 msgr2=0x7f6c4c1024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c101060 0x7f6c4c1024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 -- 172.21.15.64:0/2282721854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0fff00 msgr2=0x7f6c4c103de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c103de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 -- 172.21.15.64:0/2282721854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400095f0 con 0x7f6c4c101a20 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c50fa9700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c517aa700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 0x7f6c4c102a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400ef30 tx=0x7f6c4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44018070 con 0x7f6c4c101a20 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c4400f040 con 0x7f6c4c101a20 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c1a1220 con 0x7f6c4c101a20 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400cd00 con 0x7f6c4c101a20 2024-05-08T21:49:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.495+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c1a16f0 con 0x7f6c4c101a20 2024-05-08T21:49:14.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.496+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f6c44004000 con 0x7f6c4c101a20 2024-05-08T21:49:14.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.496+0000 7f6c49ffb700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6c3c0389d0 0x7f6c3c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:14.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.496+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c4404ae70 con 0x7f6c4c101a20 2024-05-08T21:49:14.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.496+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c04ec60 con 0x7f6c4c101a20 2024-05-08T21:49:14.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.497+0000 7f6c50fa9700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6c3c0389d0 0x7f6c3c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:14.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.497+0000 7f6c50fa9700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6c3c0389d0 0x7f6c3c03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b2e0 tx=0x7f6c40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.502+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6c440042b0 con 0x7f6c4c101a20 2024-05-08T21:49:14.641 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:14.165+0000 7f24d6d11200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:49:14.641 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:14.314+0000 7f24d6d11200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:49:14.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:14.716+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c4c069060 con 0x7f6c3c0389d0 2024-05-08T21:49:14.742 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:14 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:14.350+0000 7f98f0a9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:49:15.025 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:14 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:14.741+0000 7f98f0a9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:14 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:14.933+0000 7f98f0a9d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:15 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:15.024+0000 7f98f0a9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: Reconfiguring daemon mgr.x on smithi064 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:15.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.027 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:15.027 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:15.027 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:15.141 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:14.955+0000 7f24d6d11200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:49:15.141 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:15.138+0000 7f24d6d11200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:49:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T21:49:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: Reconfiguring daemon mgr.x on smithi064 2024-05-08T21:49:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:15.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: Reconfiguring daemon mgr.x on smithi064 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:15.641 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:15.332+0000 7f24d6d11200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:49:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:15 smithi160 ceph-mon[46962]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:15 smithi160 ceph-mon[46962]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:16.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:16.045 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:15 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:15.787+0000 7f98f0a9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:49:16.045 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:15 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:15.885+0000 7f98f0a9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:49:16.045 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:16 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:16.033+0000 7f98f0a9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:49:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:15 smithi186 ceph-mon[46619]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:15 smithi186 ceph-mon[46619]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:15 smithi064 ceph-mon[42500]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:15 smithi064 ceph-mon[42500]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:16.554 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:16.265+0000 7f24d6d11200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:49:16.833 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:16.553+0000 7f24d6d11200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:49:16.833 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:16.642+0000 7f24d6d11200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:49:16.833 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 21:49:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z[48481]: 2024-05-08T21:49:16.832+0000 7f24d6d11200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:49:17.036 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:16 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:16.659+0000 7f98f0a9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:49:17.036 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:16 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:16.840+0000 7f98f0a9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1056618302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bb0a350-33bc-4f0d-a340-96bb54726c36"}]: dispatch 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1056618302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bb0a350-33bc-4f0d-a340-96bb54726c36"}]': finished 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: Standby manager daemon z started 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T21:49:17.037 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:16 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:49:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1056618302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bb0a350-33bc-4f0d-a340-96bb54726c36"}]: dispatch 2024-05-08T21:49:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1056618302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bb0a350-33bc-4f0d-a340-96bb54726c36"}]': finished 2024-05-08T21:49:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T21:49:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: Standby manager daemon z started 2024-05-08T21:49:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T21:49:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:49:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T21:49:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:16 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1056618302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bb0a350-33bc-4f0d-a340-96bb54726c36"}]: dispatch 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1056618302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bb0a350-33bc-4f0d-a340-96bb54726c36"}]': finished 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: Standby manager daemon z started 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T21:49:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:16 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.186:0/3899497875' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:49:17.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:17.639+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f6c440058f0 con 0x7f6c4c101a20 2024-05-08T21:49:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:17 smithi064 ceph-mon[42500]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:17 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1254788991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:17 smithi064 ceph-mon[42500]: mgrmap e14: x(active, since 57s), standbys: z 2024-05-08T21:49:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T21:49:18.249 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:17.035+0000 7f98f0a9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:49:18.249 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:17.966+0000 7f98f0a9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:49:18.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:17 smithi160 ceph-mon[46962]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:18.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:17 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1254788991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:18.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:17 smithi160 ceph-mon[46962]: mgrmap e14: x(active, since 57s), standbys: z 2024-05-08T21:49:18.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T21:49:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:17 smithi186 ceph-mon[46619]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:17 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1254788991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:17 smithi186 ceph-mon[46619]: mgrmap e14: x(active, since 57s), standbys: z 2024-05-08T21:49:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T21:49:18.526 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:18.247+0000 7f98f0a9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:49:18.526 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:18.335+0000 7f98f0a9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:49:18.526 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 21:49:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y[48447]: 2024-05-08T21:49:18.525+0000 7f98f0a9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:49:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:18.913+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f6c4401b320 con 0x7f6c4c101a20 2024-05-08T21:49:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:18 smithi064 ceph-mon[42500]: Standby manager daemon y started 2024-05-08T21:49:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:18 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T21:49:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:18 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:49:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:18 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T21:49:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:18 smithi064 ceph-mon[42500]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:49:19.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:18 smithi160 ceph-mon[46962]: Standby manager daemon y started 2024-05-08T21:49:19.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:18 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T21:49:19.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:18 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:49:19.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:18 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T21:49:19.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:18 smithi160 ceph-mon[46962]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:49:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:18 smithi186 ceph-mon[46619]: Standby manager daemon y started 2024-05-08T21:49:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:18 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T21:49:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:18 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:49:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:18 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T21:49:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:18 smithi186 ceph-mon[46619]: from='mgr.? 172.21.15.160:0/204255625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:49:20.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:19 smithi064 ceph-mon[42500]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:19 smithi064 ceph-mon[42500]: mgrmap e15: x(active, since 58s), standbys: z, y 2024-05-08T21:49:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T21:49:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:19 smithi160 ceph-mon[46962]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:19 smithi160 ceph-mon[46962]: mgrmap e15: x(active, since 58s), standbys: z, y 2024-05-08T21:49:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T21:49:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:19 smithi186 ceph-mon[46619]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:19 smithi186 ceph-mon[46619]: mgrmap e15: x(active, since 58s), standbys: z, y 2024-05-08T21:49:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T21:49:22.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:21 smithi064 ceph-mon[42500]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:22.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:21 smithi160 ceph-mon[46962]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:22.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:21 smithi186 ceph-mon[46619]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:23.201 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T21:49:23.201 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:23.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:22 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T21:49:23.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:22 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:23.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:22 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T21:49:23.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:22 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:24.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:23 smithi064 ceph-mon[42500]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:24.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:23 smithi064 ceph-mon[42500]: Deploying daemon osd.0 on smithi064 2024-05-08T21:49:24.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:23 smithi160 ceph-mon[46962]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:24.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:23 smithi160 ceph-mon[46962]: Deploying daemon osd.0 on smithi064 2024-05-08T21:49:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:23 smithi186 ceph-mon[46619]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:23 smithi186 ceph-mon[46619]: Deploying daemon osd.0 on smithi064 2024-05-08T21:49:26.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:25 smithi064 ceph-mon[42500]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:26.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:25 smithi160 ceph-mon[46962]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:26.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:25 smithi186 ceph-mon[46619]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:27.428 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:27 smithi064 ceph-mon[42500]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:27.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:27.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:27.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:27 smithi186 ceph-mon[46619]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:27.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:27 smithi160 ceph-mon[46962]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:27.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:27.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:27.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.410+0000 7f6c49ffb700 1 -- 172.21.15.64:0/2282721854 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c4c069060 con 0x7f6c3c0389d0 2024-05-08T21:49:28.411 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2024-05-08T21:49:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6c3c0389d0 msgr2=0x7f6c3c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6c3c0389d0 0x7f6c3c03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b2e0 tx=0x7f6c40005fb0 comp rx=0 tx=0).stop 2024-05-08T21:49:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 msgr2=0x7f6c4c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 0x7f6c4c102a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400ef30 tx=0x7f6c4400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:49:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 shutdown_connections 2024-05-08T21:49:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6c3c0389d0 0x7f6c3c03ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0fff00 0x7f6c4c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4c101a20 0x7f6c4c102a10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 --2- 172.21.15.64:0/2282721854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c101060 0x7f6c4c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.412+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 >> 172.21.15.64:0/2282721854 conn(0x7f6c4c0fb4a0 msgr2=0x7f6c4c068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.413+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 shutdown_connections 2024-05-08T21:49:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:49:28.413+0000 7f6c5320d700 1 -- 172.21.15.64:0/2282721854 wait complete. 2024-05-08T21:49:28.841 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T21:49:28.843 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_4... 2024-05-08T21:49:28.843 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T21:49:28.875 INFO:journalctl@ceph.osd.0.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:49:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:29.072 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:49:29.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:29.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:29 smithi160 ceph-mon[46962]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:30.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:29 smithi186 ceph-mon[46619]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:29 smithi064 ceph-mon[42500]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:30.399 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T21:49:30.400 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T21:49:30.400 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T21:49:30.400 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T21:49:30.400 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126733 s, 827 MB/s 2024-05-08T21:49:30.400 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T21:49:30.666 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:49:30 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0[61096]: 2024-05-08T21:49:30.339+0000 7f2f3364a7c0 -1 osd.0 0 log_to_monitors true 2024-05-08T21:49:30.793 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-05-08T21:49:30.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:30 smithi064 ceph-mon[42500]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T21:49:31.009 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:49:31.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:30 smithi160 ceph-mon[46962]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T21:49:31.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:30 smithi186 ceph-mon[46619]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T21:49:32.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 -- 172.21.15.160:0/659478863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 msgr2=0x7f6e981014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:32.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/659478863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981014e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009990 tx=0x7f6e84009ca0 comp rx=0 tx=0).stop 2024-05-08T21:49:32.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 -- 172.21.15.160:0/659478863 shutdown_connections 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/659478863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e98101a20 0x7f6e98108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/659478863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980fff00 0x7f6e98100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/659478863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981014e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.033+0000 7f6e9f25a700 1 -- 172.21.15.160:0/659478863 >> 172.21.15.160:0/659478863 conn(0x7f6e980fb4a0 msgr2=0x7f6e980fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 -- 172.21.15.160:0/659478863 shutdown_connections 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 -- 172.21.15.160:0/659478863 wait complete. 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 Processor -- start 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 -- start start 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980fff00 0x7f6e98196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e98101a20 0x7f6e9819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819eaa0 con 0x7f6e98101a20 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819ebe0 con 0x7f6e98101060 2024-05-08T21:49:32.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.034+0000 7f6e9f25a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819ed70 con 0x7f6e980fff00 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54430/0 (socket says 172.21.15.160:54430) 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 -- 172.21.15.160:0/2056175391 learned_addr learned my addr 172.21.15.160:0/2056175391 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e9cff6700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980fff00 0x7f6e98196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e9d7f7700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e98101a20 0x7f6e9819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 -- 172.21.15.160:0/2056175391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980fff00 msgr2=0x7f6e98196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980fff00 0x7f6e98196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 -- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e98101a20 msgr2=0x7f6e9819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e98101a20 0x7f6e9819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 -- 172.21.15.160:0/2056175391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840095f0 con 0x7f6e98101060 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e97fff700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981974a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e84005910 tx=0x7f6e84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8401d070 con 0x7f6e98101060 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e84003f60 con 0x7f6e98101060 2024-05-08T21:49:32.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84022ca0 con 0x7f6e98101060 2024-05-08T21:49:32.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9819eff0 con 0x7f6e98101060 2024-05-08T21:49:32.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.035+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9819f3a0 con 0x7f6e98101060 2024-05-08T21:49:32.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.036+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9804ec60 con 0x7f6e98101060 2024-05-08T21:49:32.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.038+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f6e84003aa0 con 0x7f6e98101060 2024-05-08T21:49:32.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.038+0000 7f6e95ffb700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e880a1110 0x7f6e880a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:32.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.038+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f6e840cc7d0 con 0x7f6e98101060 2024-05-08T21:49:32.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.038+0000 7f6e9cff6700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e880a1110 0x7f6e880a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:32.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.039+0000 7f6e9cff6700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e880a1110 0x7f6e880a35d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005950 tx=0x7f6e8c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:32.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.042+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e84097b80 con 0x7f6e98101060 2024-05-08T21:49:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T21:49:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T21:49:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T21:49:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T21:49:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:32.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:32.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:32.243+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e980fc250 con 0x7f6e880a1110 2024-05-08T21:49:32.963 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:49:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0[61096]: 2024-05-08T21:49:32.592+0000 7f2f284ad700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T21:49:32.963 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:49:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0[61096]: 2024-05-08T21:49:32.600+0000 7f2f20a9b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi064 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi064 to 17114M 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:32 smithi064 ceph-mon[42500]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi064 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi064 to 17114M 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:32.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:32.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:32.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:32 smithi160 ceph-mon[46962]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi064 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi064 to 17114M 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:33.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:33.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:33.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:32 smithi186 ceph-mon[46619]: from='osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663]' entity='osd.0' 2024-05-08T21:49:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] boot 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T21:49:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:49:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:49:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:34.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:34.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] boot 2024-05-08T21:49:34.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T21:49:34.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: osd.0 [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] boot 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T21:49:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:49:35.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:34 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/3839254811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]: dispatch 2024-05-08T21:49:35.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:34 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]: dispatch 2024-05-08T21:49:35.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:34 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]': finished 2024-05-08T21:49:35.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:34 smithi160 ceph-mon[46962]: osdmap e9: 2 total, 1 up, 2 in 2024-05-08T21:49:35.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:34 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/3839254811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]: dispatch 2024-05-08T21:49:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:34 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]: dispatch 2024-05-08T21:49:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:34 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]': finished 2024-05-08T21:49:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:34 smithi186 ceph-mon[46619]: osdmap e9: 2 total, 1 up, 2 in 2024-05-08T21:49:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:34 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/3839254811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]: dispatch 2024-05-08T21:49:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:34 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]: dispatch 2024-05-08T21:49:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:34 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fb69147-6892-4a82-8c67-94919896a9ac"}]': finished 2024-05-08T21:49:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:34 smithi064 ceph-mon[42500]: osdmap e9: 2 total, 1 up, 2 in 2024-05-08T21:49:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:36.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:35 smithi160 ceph-mon[46962]: pgmap v35: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:36.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:35 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/309221016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:36.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:35 smithi160 ceph-mon[46962]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T21:49:36.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:35 smithi186 ceph-mon[46619]: pgmap v35: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:35 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/309221016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:35 smithi186 ceph-mon[46619]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T21:49:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:35 smithi064 ceph-mon[42500]: pgmap v35: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:35 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/309221016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:35 smithi064 ceph-mon[42500]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T21:49:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:37 smithi160 ceph-mon[46962]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:38.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:37 smithi186 ceph-mon[46619]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:37 smithi064 ceph-mon[42500]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:39.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:39 smithi160 ceph-mon[46962]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:39 smithi186 ceph-mon[46619]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:39 smithi064 ceph-mon[42500]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:41.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:40 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T21:49:41.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:40 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:41.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:40 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T21:49:41.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:40 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:41.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T21:49:41.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:42.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:41 smithi160 ceph-mon[46962]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:42.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:41 smithi160 ceph-mon[46962]: Deploying daemon osd.1 on smithi160 2024-05-08T21:49:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:41 smithi186 ceph-mon[46619]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:41 smithi186 ceph-mon[46619]: Deploying daemon osd.1 on smithi160 2024-05-08T21:49:42.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:41 smithi064 ceph-mon[42500]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:41 smithi064 ceph-mon[42500]: Deploying daemon osd.1 on smithi160 2024-05-08T21:49:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:43 smithi160 ceph-mon[46962]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:43 smithi186 ceph-mon[46619]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:43 smithi064 ceph-mon[42500]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:44.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:44 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:44.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:44 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:44.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:44 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:45.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:45.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:45.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:45.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:44 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:45.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:44 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:45.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:44 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:46.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:45 smithi160 ceph-mon[46962]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:46.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:45 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:45 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.099 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:45 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:46.099 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:45 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:46.099 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:45 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.202+0000 7f6e95ffb700 1 -- 172.21.15.160:0/2056175391 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6e980fc250 con 0x7f6e880a1110 2024-05-08T21:49:46.203 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-05-08T21:49:46.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e880a1110 msgr2=0x7f6e880a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:46.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e880a1110 0x7f6e880a35d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005950 tx=0x7f6e8c00b410 comp rx=0 tx=0).stop 2024-05-08T21:49:46.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 msgr2=0x7f6e981974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:46.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981974a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e84005910 tx=0x7f6e84004730 comp rx=0 tx=0).stop 2024-05-08T21:49:46.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 shutdown_connections 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6e880a1110 0x7f6e880a35d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e98101a20 0x7f6e9819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980fff00 0x7f6e98196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 --2- 172.21.15.160:0/2056175391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e98101060 0x7f6e981974a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 >> 172.21.15.160:0/2056175391 conn(0x7f6e980fb4a0 msgr2=0x7f6e980fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 shutdown_connections 2024-05-08T21:49:46.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:49:46.204+0000 7f6e9f25a700 1 -- 172.21.15.160:0/2056175391 wait complete. 2024-05-08T21:49:46.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:45 smithi064 ceph-mon[42500]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:45 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:45 smithi186 ceph-mon[46619]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:45 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:45 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:45 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:45 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:45 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:46.613 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T21:49:46.616 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-05-08T21:49:46.616 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T21:49:46.648 INFO:journalctl@ceph.osd.1.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:49:46.843 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:49:47.270 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:47 smithi186 ceph-mon[46619]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:47.270 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:47.270 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:47.270 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:47.378 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:47 smithi160 ceph-mon[46962]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:47.378 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:47.378 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:47.378 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:47.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:47 smithi064 ceph-mon[42500]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:47.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:47.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:47.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:49:48.123 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T21:49:48.124 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T21:49:48.124 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-08T21:49:48.124 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-08T21:49:48.124 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011204 s, 936 MB/s 2024-05-08T21:49:48.124 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-08T21:49:48.201 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:49:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1[51980]: 2024-05-08T21:49:47.914+0000 7fa0fe6637c0 -1 osd.1 0 log_to_monitors true 2024-05-08T21:49:48.448 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:48 smithi186 ceph-mon[46619]: from='osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:49:48.448 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:48 smithi186 ceph-mon[46619]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:49:48.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:48 smithi064 ceph-mon[42500]: from='osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:49:48.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:48 smithi064 ceph-mon[42500]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:49:48.466 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-05-08T21:49:48.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:48 smithi160 ceph-mon[46962]: from='osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:49:48.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:48 smithi160 ceph-mon[46962]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:49:48.689 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:49:49.462 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi160 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi160 to 17114M 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:49.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi160 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:49:49.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi160 to 17114M 2024-05-08T21:49:49.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:49.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:49.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.545 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:49:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1[51980]: 2024-05-08T21:49:49.209+0000 7fa0f34c6700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T21:49:49.545 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:49:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1[51980]: 2024-05-08T21:49:49.213+0000 7fa0efabc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi160 2024-05-08T21:49:49.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi160 to 17114M 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:49:49.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:49:49.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 -- 172.21.15.186:0/305046067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 msgr2=0x7fc49c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 --2- 172.21.15.186:0/305046067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1014e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009990 tx=0x7fc48c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 -- 172.21.15.186:0/305046067 shutdown_connections 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 --2- 172.21.15.186:0/305046067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c101a20 0x7fc49c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 --2- 172.21.15.186:0/305046067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c0fff00 0x7fc49c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 --2- 172.21.15.186:0/305046067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1014e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 -- 172.21.15.186:0/305046067 >> 172.21.15.186:0/305046067 conn(0x7fc49c0fb4a0 msgr2=0x7fc49c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 -- 172.21.15.186:0/305046067 shutdown_connections 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.628+0000 7fc4a193a700 1 -- 172.21.15.186:0/305046067 wait complete. 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 Processor -- start 2024-05-08T21:49:49.631 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 -- start start 2024-05-08T21:49:49.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c0fff00 0x7fc49c103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:49.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:49.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c101a20 0x7fc49c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:49.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1031b0 con 0x7fc49c101a20 2024-05-08T21:49:49.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c103320 con 0x7fc49c0fff00 2024-05-08T21:49:49.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.629+0000 7fc4a193a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1a0f40 con 0x7fc49c101060 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58438/0 (socket says 172.21.15.186:58438) 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 -- 172.21.15.186:0/491026545 learned_addr learned my addr 172.21.15.186:0/491026545 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49affd700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c0fff00 0x7fc49c103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49b7fe700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c101a20 0x7fc49c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 -- 172.21.15.186:0/491026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c0fff00 msgr2=0x7fc49c103e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c0fff00 0x7fc49c103e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 -- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c101a20 msgr2=0x7fc49c102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c101a20 0x7fc49c102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 -- 172.21.15.186:0/491026545 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c0095f0 con 0x7fc49c101060 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49b7fe700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c101a20 0x7fc49c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49affd700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c0fff00 0x7fc49c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:49:49.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc49a7fc700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1024d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0095c0 tx=0x7fc48c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:49.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c01d070 con 0x7fc49c101060 2024-05-08T21:49:49.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc48c00fb80 con 0x7fc49c101060 2024-05-08T21:49:49.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c017650 con 0x7fc49c101060 2024-05-08T21:49:49.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c1a1170 con 0x7fc49c101060 2024-05-08T21:49:49.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.630+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1a1590 con 0x7fc49c101060 2024-05-08T21:49:49.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.631+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c04ec60 con 0x7fc49c101060 2024-05-08T21:49:49.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.632+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fc48c022470 con 0x7fc49c101060 2024-05-08T21:49:49.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.633+0000 7fc4a0938700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc4840a1090 0x7fc4840a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:49:49.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.633+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7fc48c0cc880 con 0x7fc49c101060 2024-05-08T21:49:49.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.633+0000 7fc49affd700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc4840a1090 0x7fc4840a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:49:49.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.633+0000 7fc49affd700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc4840a1090 0x7fc4840a3550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc490005fd0 tx=0x7fc490005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:49:49.638 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.636+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc48c097a10 con 0x7fc49c101060 2024-05-08T21:49:49.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:49:49.835+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc49c0fd6c0 con 0x7fc4840a1090 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:50.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:50.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:50.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:49:50.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:51.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:51 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:49:51.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:51 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:49:51.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:51 smithi160 ceph-mon[46962]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:51.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:51 smithi160 ceph-mon[46962]: osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] boot 2024-05-08T21:49:51.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:51 smithi160 ceph-mon[46962]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T21:49:51.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:51.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:51 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:49:51.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:51 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:49:51.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:51 smithi186 ceph-mon[46619]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:51.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:51 smithi186 ceph-mon[46619]: osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] boot 2024-05-08T21:49:51.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:51 smithi186 ceph-mon[46619]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T21:49:51.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:51 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:49:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:51 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:49:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:51 smithi064 ceph-mon[42500]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:49:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:51 smithi064 ceph-mon[42500]: osd.1 [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] boot 2024-05-08T21:49:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:51 smithi064 ceph-mon[42500]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T21:49:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:49:52.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:52 smithi160 ceph-mon[46962]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T21:49:52.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:52 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/4018417488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]: dispatch 2024-05-08T21:49:52.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:52 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]: dispatch 2024-05-08T21:49:52.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:52 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]': finished 2024-05-08T21:49:52.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:52 smithi160 ceph-mon[46962]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T21:49:52.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:49:52.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:52 smithi186 ceph-mon[46619]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T21:49:52.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:52 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/4018417488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]: dispatch 2024-05-08T21:49:52.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:52 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]: dispatch 2024-05-08T21:49:52.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:52 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]': finished 2024-05-08T21:49:52.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:52 smithi186 ceph-mon[46619]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T21:49:52.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:49:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:52 smithi064 ceph-mon[42500]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T21:49:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:52 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/4018417488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]: dispatch 2024-05-08T21:49:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:52 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]: dispatch 2024-05-08T21:49:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:52 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65e106b3-aa74-4293-98f6-09717bbfca26"}]': finished 2024-05-08T21:49:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:52 smithi064 ceph-mon[42500]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T21:49:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:49:53.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:53 smithi160 ceph-mon[46962]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:53.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:53 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/3681865792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:53.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:53 smithi186 ceph-mon[46619]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:53.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:53 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/3681865792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:53.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:53 smithi064 ceph-mon[42500]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:53.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:53 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/3681865792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:49:55.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:55 smithi160 ceph-mon[46962]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:55.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:55 smithi186 ceph-mon[46619]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:55.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:55 smithi064 ceph-mon[42500]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:57.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:57 smithi160 ceph-mon[46962]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:57.562 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:57 smithi186 ceph-mon[46619]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:57.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:57 smithi064 ceph-mon[42500]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:58.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T21:49:58.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:58.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:58 smithi160 ceph-mon[46962]: Deploying daemon osd.2 on smithi186 2024-05-08T21:49:58.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T21:49:58.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:58.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:58 smithi186 ceph-mon[46619]: Deploying daemon osd.2 on smithi186 2024-05-08T21:49:58.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T21:49:58.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:49:58.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:58 smithi064 ceph-mon[42500]: Deploying daemon osd.2 on smithi186 2024-05-08T21:49:59.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:49:59 smithi186 ceph-mon[46619]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:59.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:49:59 smithi160 ceph-mon[46962]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:49:59.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:49:59 smithi064 ceph-mon[42500]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:00.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:00 smithi186 ceph-mon[46619]: overall HEALTH_OK 2024-05-08T21:50:00.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:00 smithi160 ceph-mon[46962]: overall HEALTH_OK 2024-05-08T21:50:00.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:00 smithi064 ceph-mon[42500]: overall HEALTH_OK 2024-05-08T21:50:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:01 smithi186 ceph-mon[46619]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:01 smithi160 ceph-mon[46962]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:01.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:01 smithi064 ceph-mon[42500]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:01.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:01.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:01.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:03.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.441+0000 7fc4a0938700 1 -- 172.21.15.186:0/491026545 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc49c0fd6c0 con 0x7fc4840a1090 2024-05-08T21:50:03.441 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-05-08T21:50:03.444 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc4840a1090 msgr2=0x7fc4840a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:03.444 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc4840a1090 0x7fc4840a3550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc490005fd0 tx=0x7fc490005ee0 comp rx=0 tx=0).stop 2024-05-08T21:50:03.444 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 msgr2=0x7fc49c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:03.444 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1024d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0095c0 tx=0x7fc48c00f690 comp rx=0 tx=0).stop 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 shutdown_connections 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc4840a1090 0x7fc4840a3550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc49c0fff00 0x7fc49c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc49c101a20 0x7fc49c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 --2- 172.21.15.186:0/491026545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc49c101060 0x7fc49c1024d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 >> 172.21.15.186:0/491026545 conn(0x7fc49c0fb4a0 msgr2=0x7fc49c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 shutdown_connections 2024-05-08T21:50:03.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:03.444+0000 7fc4a193a700 1 -- 172.21.15.186:0/491026545 wait complete. 2024-05-08T21:50:03.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:03 smithi186 ceph-mon[46619]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:03.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:03.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:03.609 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:03 smithi064 ceph-mon[42500]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:03 smithi160 ceph-mon[46962]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:03.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:03.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:03.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:03.831 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T21:50:03.833 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/vg_nvme/lv_3... 2024-05-08T21:50:03.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T21:50:03.864 INFO:journalctl@ceph.osd.2.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:50:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:50:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:04 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:04 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:04 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:04.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:04 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:04.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:04 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:04.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:04 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:04.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T21:50:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T21:50:05.387 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-08T21:50:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-08T21:50:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111688 s, 939 MB/s 2024-05-08T21:50:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-08T21:50:05.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:05 smithi186 ceph-mon[46619]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:05 smithi186 ceph-mon[46619]: from='osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:50:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:05 smithi186 ceph-mon[46619]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:50:05.470 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:05 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2[52434]: 2024-05-08T21:50:05.169+0000 7f39dd1667c0 -1 osd.2 0 log_to_monitors true 2024-05-08T21:50:05.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:05 smithi064 ceph-mon[42500]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:05.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:05 smithi064 ceph-mon[42500]: from='osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:50:05.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:05 smithi064 ceph-mon[42500]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:50:05.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-05-08T21:50:05.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:05 smithi160 ceph-mon[46962]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:05.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:05 smithi160 ceph-mon[46962]: from='osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:50:05.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:05 smithi160 ceph-mon[46962]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:50:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:50:06.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T21:50:06.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T21:50:06.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:06.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:06.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi186 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi186 to 17114M 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:06.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi186 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi186 to 17114M 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:06.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.891 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:06 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2[52434]: 2024-05-08T21:50:06.433+0000 7f39d1fc9700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T21:50:06.891 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:06 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2[52434]: 2024-05-08T21:50:06.438+0000 7f39ce5bf700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:50:06.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T21:50:06.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi186 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi186 to 17114M 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:06.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 -- 172.21.15.64:0/2371229232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb881043c0 msgr2=0x7fbb881067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2371229232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb881043c0 0x7fbb881067b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009990 tx=0x7fbb78009ca0 comp rx=0 tx=0).stop 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 -- 172.21.15.64:0/2371229232 shutdown_connections 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2371229232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb88101a90 0x7fbb88103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2371229232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb88106cf0 0x7fbb88109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2371229232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb881043c0 0x7fbb881067b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 -- 172.21.15.64:0/2371229232 >> 172.21.15.64:0/2371229232 conn(0x7fbb880fb4a0 msgr2=0x7fbb880fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 -- 172.21.15.64:0/2371229232 shutdown_connections 2024-05-08T21:50:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.969+0000 7fbb8f683700 1 -- 172.21.15.64:0/2371229232 wait complete. 2024-05-08T21:50:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 Processor -- start 2024-05-08T21:50:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 -- start start 2024-05-08T21:50:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb88101a90 0x7fbb88196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb881043c0 0x7fbb881974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 0x7fbb8819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8819eaf0 con 0x7fbb88106cf0 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8819ec30 con 0x7fbb88101a90 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8f683700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8819edc0 con 0x7fbb881043c0 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.970+0000 7fbb8dc20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 0x7fbb8819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 0x7fbb8819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41542/0 (socket says 172.21.15.64:41542) 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 -- 172.21.15.64:0/2716924770 learned_addr learned my addr 172.21.15.64:0/2716924770 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8d41f700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb88101a90 0x7fbb88196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8cc1e700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb881043c0 0x7fbb881974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 -- 172.21.15.64:0/2716924770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb881043c0 msgr2=0x7fbb881974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb881043c0 0x7fbb881974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 -- 172.21.15.64:0/2716924770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb88101a90 msgr2=0x7fbb88196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb88101a90 0x7fbb88196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 -- 172.21.15.64:0/2716924770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780095f0 con 0x7fbb88106cf0 2024-05-08T21:50:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8dc20700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 0x7fbb8819e520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb80007ef0 tx=0x7fbb8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:06.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80017070 con 0x7fbb88106cf0 2024-05-08T21:50:06.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb8000f040 con 0x7fbb88106cf0 2024-05-08T21:50:06.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8819f0a0 con 0x7fbb88106cf0 2024-05-08T21:50:06.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004280 con 0x7fbb88106cf0 2024-05-08T21:50:06.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.971+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8819f550 con 0x7fbb88106cf0 2024-05-08T21:50:06.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.972+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb880fd020 con 0x7fbb88106cf0 2024-05-08T21:50:06.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.974+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fbb8000db20 con 0x7fbb88106cf0 2024-05-08T21:50:06.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.975+0000 7fbb7e7fc700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb740a1090 0x7fbb740a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:06.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.975+0000 7fbb8d41f700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb740a1090 0x7fbb740a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:06.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.975+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fbb800cad30 con 0x7fbb88106cf0 2024-05-08T21:50:06.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.975+0000 7fbb8d41f700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb740a1090 0x7fbb740a3550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009a10 tx=0x7fbb84009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:06.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:06.978+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbb80095ca0 con 0x7fbb88106cf0 2024-05-08T21:50:07.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:07.192+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb880613e0 con 0x7fbb740a1090 2024-05-08T21:50:07.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:07.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:50:07.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T21:50:07.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:07.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:07.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:07.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:07.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:50:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T21:50:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:07.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:07.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:07.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:07.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:50:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T21:50:07.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:07.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:07.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:07.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:07.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:08 smithi186 ceph-mon[46619]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:08 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:08 smithi186 ceph-mon[46619]: osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] boot 2024-05-08T21:50:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:08 smithi186 ceph-mon[46619]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T21:50:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:08 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:08 smithi064 ceph-mon[42500]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:08 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:08 smithi064 ceph-mon[42500]: osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] boot 2024-05-08T21:50:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:08 smithi064 ceph-mon[42500]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T21:50:08.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:08 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:09.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:08 smithi160 ceph-mon[46962]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:09.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:08 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:09.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:08 smithi160 ceph-mon[46962]: osd.2 [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] boot 2024-05-08T21:50:09.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:08 smithi160 ceph-mon[46962]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T21:50:09.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:08 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:50:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:50:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/716235667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a840d5a-a687-4cf2-a946-7a2f46bdfd26"}]: dispatch 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/716235667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a840d5a-a687-4cf2-a946-7a2f46bdfd26"}]': finished 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T21:50:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:09.963 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:09 smithi064 sudo[67552]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:50:09.963 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:09 smithi064 sudo[67552]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:09.963 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:09 smithi064 sudo[67552]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:09.963 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:09 smithi064 sudo[67552]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:50:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:50:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:50:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/716235667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a840d5a-a687-4cf2-a946-7a2f46bdfd26"}]: dispatch 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/716235667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a840d5a-a687-4cf2-a946-7a2f46bdfd26"}]': finished 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T21:50:09.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/716235667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a840d5a-a687-4cf2-a946-7a2f46bdfd26"}]: dispatch 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/716235667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a840d5a-a687-4cf2-a946-7a2f46bdfd26"}]': finished 2024-05-08T21:50:10.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T21:50:10.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:10.462 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:10 smithi064 sudo[67580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:50:10.463 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:10 smithi064 sudo[67580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:10.463 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:10 smithi064 sudo[67580]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:10.463 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 21:50:10 smithi064 sudo[67580]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:10.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:10 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1283993421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:10 smithi160 ceph-mon[46962]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T21:50:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:10.795 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:50:10.795 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:10.795 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56804]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:10.795 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56804]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:10.795 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:50:10.795 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:10.796 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56807]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:10.796 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 21:50:10 smithi160 sudo[56807]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:10.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:10 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1283993421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:10.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:10 smithi186 ceph-mon[46619]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T21:50:10.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:10.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:10 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1283993421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:10 smithi064 ceph-mon[42500]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T21:50:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:10 smithi186 sudo[57271]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:10 smithi186 sudo[57271]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:10 smithi186 sudo[57271]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:10 smithi186 sudo[57271]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:11 smithi186 sudo[57274]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:11 smithi186 sudo[57274]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:11.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:11 smithi186 sudo[57274]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:11.142 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 21:50:11 smithi186 sudo[57274]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:11 smithi064 sudo[68358]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:50:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:11 smithi064 sudo[68358]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:11 smithi064 sudo[68358]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:11 smithi064 sudo[68358]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:12.073+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbb80096070 con 0x7fbb88106cf0 2024-05-08T21:50:12.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 sudo[68451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:50:12.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 sudo[68451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:12.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 sudo[68451]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:12.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 sudo[68451]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:12.551 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:50:12.551 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:12.551 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57277]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:12.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57277]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:12.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57280]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:50:12.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:12.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57280]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:12.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 sudo[57280]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 ceph-mon[46619]: pgmap v65: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:12 smithi186 ceph-mon[46619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 ceph-mon[42500]: pgmap v65: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:12 smithi064 ceph-mon[42500]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 ceph-mon[46962]: pgmap v65: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 ceph-mon[46962]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 sudo[56810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 sudo[56810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 sudo[56810]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:13.031 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:12 smithi160 sudo[56810]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:13.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 sudo[56813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:50:13.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 sudo[56813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:50:13.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 sudo[56813]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:50:13.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 sudo[56813]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: mgrmap e16: x(active, since 112s), standbys: z, y 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:13.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:13.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:13 smithi064 ceph-mon[42500]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: mgrmap e16: x(active, since 112s), standbys: z, y 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:13.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:13 smithi186 ceph-mon[46619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:14.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: mgrmap e16: x(active, since 112s), standbys: z, y 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:50:14.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:13 smithi160 ceph-mon[46962]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:50:15.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:15 smithi064 ceph-mon[42500]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:15.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T21:50:15.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:15 smithi186 ceph-mon[46619]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T21:50:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:15 smithi160 ceph-mon[46962]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:16.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T21:50:16.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:16.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:16 smithi064 ceph-mon[42500]: Deploying daemon osd.3 on smithi064 2024-05-08T21:50:16.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:16 smithi186 ceph-mon[46619]: Deploying daemon osd.3 on smithi064 2024-05-08T21:50:17.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:16 smithi160 ceph-mon[46962]: Deploying daemon osd.3 on smithi064 2024-05-08T21:50:17.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:17 smithi064 ceph-mon[42500]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:17.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:17 smithi186 ceph-mon[46619]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:18.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:17 smithi160 ceph-mon[46962]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:19 smithi064 ceph-mon[42500]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-08T21:50:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:19 smithi186 ceph-mon[46619]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-08T21:50:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:20.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:19 smithi160 ceph-mon[46962]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-08T21:50:20.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:20.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:20.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:20.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.733+0000 7fbb7e7fc700 1 -- 172.21.15.64:0/2716924770 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbb880613e0 con 0x7fbb740a1090 2024-05-08T21:50:20.734 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb740a1090 msgr2=0x7fbb740a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb740a1090 0x7fbb740a3550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009a10 tx=0x7fbb84009190 comp rx=0 tx=0).stop 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 msgr2=0x7fbb8819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 0x7fbb8819e520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb80007ef0 tx=0x7fbb8000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 shutdown_connections 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb740a1090 0x7fbb740a3550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb88101a90 0x7fbb88196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb88106cf0 0x7fbb8819e520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:20.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 --2- 172.21.15.64:0/2716924770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb881043c0 0x7fbb881974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:20.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.735+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 >> 172.21.15.64:0/2716924770 conn(0x7fbb880fb4a0 msgr2=0x7fbb881023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:20.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.736+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 shutdown_connections 2024-05-08T21:50:20.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:20.736+0000 7fbb8f683700 1 -- 172.21.15.64:0/2716924770 wait complete. 2024-05-08T21:50:21.140 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T21:50:21.143 INFO:tasks.cephadm:Deploying osd.4 on smithi160 with /dev/vg_nvme/lv_3... 2024-05-08T21:50:21.143 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T21:50:21.177 INFO:journalctl@ceph.osd.3.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:50:21.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:21.378 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:50:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-08T21:50:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:21.498 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.498 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.498 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.498 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:22.791 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T21:50:22.791 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T21:50:22.792 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T21:50:22.792 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T21:50:22.792 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0168308 s, 623 MB/s 2024-05-08T21:50:22.792 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T21:50:22.894 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 21:50:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3[69808]: 2024-05-08T21:50:22.570+0000 7f35d1a7a7c0 -1 osd.3 0 log_to_monitors true 2024-05-08T21:50:23.179 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-05-08T21:50:23.400 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:50:23.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:23 smithi064 ceph-mon[42500]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T21:50:23.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:23 smithi064 ceph-mon[42500]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T21:50:23.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:23 smithi160 ceph-mon[46962]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T21:50:23.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:23 smithi160 ceph-mon[46962]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T21:50:23.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:23 smithi186 ceph-mon[46619]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T21:50:23.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:23 smithi186 ceph-mon[46619]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 -- 172.21.15.160:0/1539623536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100fe0 msgr2=0x7f7d48101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/1539623536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100fe0 0x7f7d48101400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0099c0 tx=0x7f7d3c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 -- 172.21.15.160:0/1539623536 shutdown_connections 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/1539623536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100fe0 0x7f7d48101400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/1539623536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48102b80 0x7f7d48109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/1539623536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d481021e0 0x7f7d48102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 -- 172.21.15.160:0/1539623536 >> 172.21.15.160:0/1539623536 conn(0x7f7d480fc580 msgr2=0x7f7d480fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 -- 172.21.15.160:0/1539623536 shutdown_connections 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.393+0000 7f7d4e09c700 1 -- 172.21.15.160:0/1539623536 wait complete. 2024-05-08T21:50:24.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.394+0000 7f7d4e09c700 1 Processor -- start 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.394+0000 7f7d4e09c700 1 -- start start 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.394+0000 7f7d4e09c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100fe0 0x7f7d48196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.394+0000 7f7d4e09c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 0x7f7d481974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d4e09c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48102b80 0x7f7d4819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d4e09c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819eac0 con 0x7f7d48100fe0 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d4e09c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819ec00 con 0x7f7d481021e0 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d4e09c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819ed90 con 0x7f7d48102b80 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 0x7f7d481974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 0x7f7d481974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56142/0 (socket says 172.21.15.160:56142) 2024-05-08T21:50:24.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 -- 172.21.15.160:0/3150628445 learned_addr learned my addr 172.21.15.160:0/3150628445 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d477fe700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100fe0 0x7f7d48196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 -- 172.21.15.160:0/3150628445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48102b80 msgr2=0x7f7d4819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48102b80 0x7f7d4819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 -- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100fe0 msgr2=0x7f7d48196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100fe0 0x7f7d48196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 -- 172.21.15.160:0/3150628445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0095f0 con 0x7f7d481021e0 2024-05-08T21:50:24.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d46ffd700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 0x7f7d481974c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800d950 tx=0x7f7d3800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:24.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38009940 con 0x7f7d481021e0 2024-05-08T21:50:24.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d38010460 con 0x7f7d481021e0 2024-05-08T21:50:24.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4819f070 con 0x7f7d481021e0 2024-05-08T21:50:24.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800f600 con 0x7f7d481021e0 2024-05-08T21:50:24.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.395+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4819f570 con 0x7f7d481021e0 2024-05-08T21:50:24.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.396+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d481084e0 con 0x7f7d481021e0 2024-05-08T21:50:24.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.397+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7d380105d0 con 0x7f7d481021e0 2024-05-08T21:50:24.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.398+0000 7f7d44ff9700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d300a1120 0x7f7d300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:24.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.398+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f7d380cb3f0 con 0x7f7d481021e0 2024-05-08T21:50:24.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.398+0000 7f7d477fe700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d300a1120 0x7f7d300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:24.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.399+0000 7f7d477fe700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d300a1120 0x7f7d300a35e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009f50 tx=0x7f7d3c005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:24.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.402+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d38095ec0 con 0x7f7d481021e0 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi064 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi064 to 8557M 2024-05-08T21:50:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:24.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:24.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:24 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T21:50:24.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi064 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi064 to 8557M 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:24 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:24.605+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d4802ce10 con 0x7f7d300a1120 2024-05-08T21:50:24.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T21:50:24.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T21:50:24.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi064 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi064 to 8557M 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:24.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:24.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:24 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:25.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:25 smithi160 ceph-mon[46962]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' 2024-05-08T21:50:25.463 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 21:50:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3[69808]: 2024-05-08T21:50:25.020+0000 7f35c68dd700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T21:50:25.463 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 21:50:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3[69808]: 2024-05-08T21:50:25.029+0000 7f35beecb700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:50:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T21:50:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:50:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T21:50:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:25.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:25.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:25.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:25.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:25 smithi064 ceph-mon[42500]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' 2024-05-08T21:50:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T21:50:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:50:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T21:50:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:25 smithi186 ceph-mon[46619]: from='osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060]' entity='osd.3' 2024-05-08T21:50:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:26 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:50:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:26 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:50:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:26 smithi064 ceph-mon[42500]: osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] boot 2024-05-08T21:50:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:26 smithi064 ceph-mon[42500]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T21:50:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:26 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:26 smithi064 ceph-mon[42500]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T21:50:26.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:26 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:50:26.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:26 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:50:26.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:26 smithi160 ceph-mon[46962]: osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] boot 2024-05-08T21:50:26.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:26 smithi160 ceph-mon[46962]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T21:50:26.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:26 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:26.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:26 smithi160 ceph-mon[46962]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T21:50:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:26 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:50:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:26 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:50:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:26 smithi186 ceph-mon[46619]: osd.3 [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] boot 2024-05-08T21:50:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:26 smithi186 ceph-mon[46619]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T21:50:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:26 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:50:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:26 smithi186 ceph-mon[46619]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T21:50:27.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:27.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/3228927064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]: dispatch 2024-05-08T21:50:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]: dispatch 2024-05-08T21:50:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]': finished 2024-05-08T21:50:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T21:50:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/4003576878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/3228927064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]: dispatch 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]: dispatch 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]': finished 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/4003576878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/3228927064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]: dispatch 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]: dispatch 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e00f8790-f664-4e52-b320-ab5e454f0235"}]': finished 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/4003576878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:28 smithi064 ceph-mon[42500]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T21:50:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:28.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:28 smithi160 ceph-mon[46962]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T21:50:28.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:28 smithi186 ceph-mon[46619]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T21:50:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:29.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:29 smithi064 ceph-mon[42500]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:29.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:29 smithi064 ceph-mon[42500]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T21:50:29.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:29.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:29 smithi160 ceph-mon[46962]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:29.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:29 smithi160 ceph-mon[46962]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T21:50:29.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:29.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:29 smithi186 ceph-mon[46619]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:29.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:29 smithi186 ceph-mon[46619]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T21:50:29.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:31 smithi064 ceph-mon[42500]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:31.780 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:31 smithi160 ceph-mon[46962]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:31.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:31 smithi186 ceph-mon[46619]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:33.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:33 smithi160 ceph-mon[46962]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-08T21:50:33.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T21:50:33.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:33.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:33 smithi160 ceph-mon[46962]: Deploying daemon osd.4 on smithi160 2024-05-08T21:50:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:33 smithi064 ceph-mon[42500]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-08T21:50:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T21:50:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:33 smithi064 ceph-mon[42500]: Deploying daemon osd.4 on smithi160 2024-05-08T21:50:33.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:33 smithi186 ceph-mon[46619]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-08T21:50:33.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T21:50:33.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:33.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:33 smithi186 ceph-mon[46619]: Deploying daemon osd.4 on smithi160 2024-05-08T21:50:35.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:35 smithi064 ceph-mon[42500]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T21:50:35.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:35 smithi160 ceph-mon[46962]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T21:50:35.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:35 smithi186 ceph-mon[46619]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T21:50:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:36.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:36 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:36.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:36 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:36.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:36 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:36.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:36 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:36.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:36 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:36.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:36 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:37 smithi160 ceph-mon[46962]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T21:50:37.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:37.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:37.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:37 smithi186 ceph-mon[46619]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T21:50:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:37 smithi064 ceph-mon[42500]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T21:50:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:38.460 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.459+0000 7f7d44ff9700 1 -- 172.21.15.160:0/3150628445 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d4802ce10 con 0x7f7d300a1120 2024-05-08T21:50:38.460 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 4 on host 'smithi160' 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d300a1120 msgr2=0x7f7d300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d300a1120 0x7f7d300a35e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009f50 tx=0x7f7d3c005ea0 comp rx=0 tx=0).stop 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 msgr2=0x7f7d481974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 0x7f7d481974c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800d950 tx=0x7f7d3800dc60 comp rx=0 tx=0).stop 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 shutdown_connections 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d300a1120 0x7f7d300a35e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48102b80 0x7f7d4819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:38.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100fe0 0x7f7d48196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:38.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 --2- 172.21.15.160:0/3150628445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481021e0 0x7f7d481974c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:38.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 >> 172.21.15.160:0/3150628445 conn(0x7f7d480fc580 msgr2=0x7f7d48105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:38.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 shutdown_connections 2024-05-08T21:50:38.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:50:38.462+0000 7f7d4e09c700 1 -- 172.21.15.160:0/3150628445 wait complete. 2024-05-08T21:50:38.840 DEBUG:teuthology.orchestra.run.smithi160:osd.4> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T21:50:38.842 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-05-08T21:50:38.842 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T21:50:38.874 INFO:journalctl@ceph.osd.4.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:50:39.078 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:50:39.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:39 smithi064 ceph-mon[42500]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:50:39.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:39.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:39.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:39.719 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:39 smithi160 ceph-mon[46962]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:50:39.719 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:39.719 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:39.720 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:39.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:39 smithi186 ceph-mon[46619]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:50:39.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:39.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:39.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:40.405 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T21:50:40.405 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T21:50:40.405 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-08T21:50:40.406 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-08T21:50:40.406 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149681 s, 701 MB/s 2024-05-08T21:50:40.406 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-08T21:50:40.544 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 21:50:40 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4[59914]: 2024-05-08T21:50:40.193+0000 7fa7f254b7c0 -1 osd.4 0 log_to_monitors true 2024-05-08T21:50:40.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:40 smithi160 ceph-mon[46962]: from='osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:50:40.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:40 smithi160 ceph-mon[46962]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:50:40.698 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:40 smithi186 ceph-mon[46619]: from='osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:50:40.698 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:40 smithi186 ceph-mon[46619]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:50:40.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:40 smithi064 ceph-mon[42500]: from='osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:50:40.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:40 smithi064 ceph-mon[42500]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:50:40.744 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-05-08T21:50:40.962 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:50:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T21:50:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T21:50:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T21:50:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:50:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:41.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:41.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:41.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.795 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 21:50:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4[59914]: 2024-05-08T21:50:41.454+0000 7fa7e73ae700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T21:50:41.795 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 21:50:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4[59914]: 2024-05-08T21:50:41.461+0000 7fa7e39a4700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:50:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T21:50:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T21:50:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T21:50:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:50:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:41.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 -- 172.21.15.186:0/1108603075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 msgr2=0x7ff3ac1002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 --2- 172.21.15.186:0/1108603075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac1002b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009990 tx=0x7ff39c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 -- 172.21.15.186:0/1108603075 shutdown_connections 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 --2- 172.21.15.186:0/1108603075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 0x7ff3ac108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 --2- 172.21.15.186:0/1108603075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac1002b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 --2- 172.21.15.186:0/1108603075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 0x7ff3ac101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.896+0000 7ff3b33fb700 1 -- 172.21.15.186:0/1108603075 >> 172.21.15.186:0/1108603075 conn(0x7ff3ac0fb4a0 msgr2=0x7ff3ac0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:41.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 -- 172.21.15.186:0/1108603075 shutdown_connections 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 -- 172.21.15.186:0/1108603075 wait complete. 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 Processor -- start 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 -- start start 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac103e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 0x7ff3ac1024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 0x7ff3ac102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1031d0 con 0x7ff3ac101080 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac103340 con 0x7ff3ac101a40 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b33fb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1a0fa0 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b1197700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac103e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b1197700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac103e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:59498/0 (socket says 172.21.15.186:59498) 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.897+0000 7ff3b1197700 1 -- 172.21.15.186:0/296669826 learned_addr learned my addr 172.21.15.186:0/296669826 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:50:41.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b0996700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 0x7ff3ac1024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1998700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 0x7ff3ac102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1197700 1 -- 172.21.15.186:0/296669826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 msgr2=0x7ff3ac102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1197700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 0x7ff3ac102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1197700 1 -- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 msgr2=0x7ff3ac1024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1197700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 0x7ff3ac1024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1197700 1 -- 172.21.15.186:0/296669826 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c0095f0 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b0996700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 0x7ff3ac1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1998700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 0x7ff3ac102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b1197700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac103e50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0095c0 tx=0x7ff39c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c01d070 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b33fb700 1 -- 172.21.15.186:0/296669826 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac1a11d0 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3b33fb700 1 -- 172.21.15.186:0/296669826 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac1a1690 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff39c00fb80 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.898+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c017650 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.903 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.899+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c0052f0 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.903 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.900+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff39c022470 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.901+0000 7ff3a27fc700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff3980a1120 0x7ff3980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:41.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.901+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7ff39c0ccbe0 con 0x7ff3ac0ffe90 2024-05-08T21:50:41.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.901+0000 7ff3b0996700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff3980a1120 0x7ff3980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:41.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.902+0000 7ff3b0996700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff3980a1120 0x7ff3980a35e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8005950 tx=0x7ff3a80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:41.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:41.906+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff39c097590 con 0x7ff3ac0ffe90 2024-05-08T21:50:42.105 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:42.104+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff38c000bc0 con 0x7ff3980a1120 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi160 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi160 to 8557M 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:42.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:42.760 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi160 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi160 to 8557M 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:42.761 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:42.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi160 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi160 to 8557M 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:50:42.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:43.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] boot 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T21:50:43.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:50:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:50:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:43.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:43.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] boot 2024-05-08T21:50:43.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T21:50:43.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:43.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: osd.4 [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] boot 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T21:50:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:50:44.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:44 smithi160 ceph-mon[46962]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T21:50:44.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:44 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/2563529543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]: dispatch 2024-05-08T21:50:44.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:44 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]: dispatch 2024-05-08T21:50:44.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:44 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]': finished 2024-05-08T21:50:44.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:44 smithi160 ceph-mon[46962]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T21:50:44.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:44 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:44.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:44 smithi186 ceph-mon[46619]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T21:50:44.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:44 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/2563529543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]: dispatch 2024-05-08T21:50:44.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:44 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]: dispatch 2024-05-08T21:50:44.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:44 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]': finished 2024-05-08T21:50:44.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:44 smithi186 ceph-mon[46619]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T21:50:44.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:44 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:44 smithi064 ceph-mon[42500]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T21:50:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:44 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/2563529543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]: dispatch 2024-05-08T21:50:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:44 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]: dispatch 2024-05-08T21:50:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:44 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e44e11d-06e2-40ef-9bf4-43bfd19c99da"}]': finished 2024-05-08T21:50:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:44 smithi064 ceph-mon[42500]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T21:50:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:44 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:45.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:45 smithi160 ceph-mon[46962]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:45.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:45 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/1825384983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:45.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:45 smithi186 ceph-mon[46619]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:45.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:45 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/1825384983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:45.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:45 smithi064 ceph-mon[42500]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:45.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:45 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/1825384983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:50:47.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:47 smithi160 ceph-mon[46962]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:47.819 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:47 smithi186 ceph-mon[46619]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:47 smithi064 ceph-mon[42500]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:49.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:49 smithi160 ceph-mon[46962]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:49.855 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:49 smithi186 ceph-mon[46619]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:49.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:49 smithi064 ceph-mon[42500]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:50.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T21:50:50.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:50 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:50.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:50 smithi186 ceph-mon[46619]: Deploying daemon osd.5 on smithi186 2024-05-08T21:50:50.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T21:50:50.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:50 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:50.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:50 smithi160 ceph-mon[46962]: Deploying daemon osd.5 on smithi186 2024-05-08T21:50:50.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T21:50:50.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:50 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:50.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:50 smithi064 ceph-mon[42500]: Deploying daemon osd.5 on smithi186 2024-05-08T21:50:51.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:51 smithi186 ceph-mon[46619]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:51.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:51 smithi160 ceph-mon[46962]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:51.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:51 smithi064 ceph-mon[42500]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:53.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:53 smithi160 ceph-mon[46962]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:53.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:53 smithi186 ceph-mon[46619]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:53.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:53 smithi064 ceph-mon[42500]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:54.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:54 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:54.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:54 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:54.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:54 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:54.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:54.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:54.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:54.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:54.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:54.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:55.763 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-05-08T21:50:55.764 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.762+0000 7ff3a27fc700 1 -- 172.21.15.186:0/296669826 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff38c000bc0 con 0x7ff3980a1120 2024-05-08T21:50:55.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff3980a1120 msgr2=0x7ff3980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:55.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff3980a1120 0x7ff3980a35e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8005950 tx=0x7ff3a80058e0 comp rx=0 tx=0).stop 2024-05-08T21:50:55.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 msgr2=0x7ff3ac103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:55.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac103e50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0095c0 tx=0x7ff39c00f690 comp rx=0 tx=0).stop 2024-05-08T21:50:55.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 shutdown_connections 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff3980a1120 0x7ff3980a35e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3ac101a40 0x7ff3ac102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac0ffe90 0x7ff3ac103e50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 --2- 172.21.15.186:0/296669826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac101080 0x7ff3ac1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 >> 172.21.15.186:0/296669826 conn(0x7ff3ac0fb4a0 msgr2=0x7ff3ac0696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 shutdown_connections 2024-05-08T21:50:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:50:55.765+0000 7ff397fff700 1 -- 172.21.15.186:0/296669826 wait complete. 2024-05-08T21:50:56.144 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T21:50:56.146 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2024-05-08T21:50:56.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.171 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:55 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.178 INFO:journalctl@ceph.osd.5.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:55 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:55 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:56.387 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:50:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T21:50:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T21:50:57.680 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-08T21:50:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-08T21:50:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129117 s, 812 MB/s 2024-05-08T21:50:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-08T21:50:57.849 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 21:50:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T21:50:57.508+0000 7fc5a1d3a7c0 -1 osd.5 0 log_to_monitors true 2024-05-08T21:50:58.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:57 smithi064 ceph-mon[42500]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:58.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:57 smithi064 ceph-mon[42500]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:50:58.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-05-08T21:50:58.120 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:57 smithi186 ceph-mon[46619]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:58.120 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:57 smithi186 ceph-mon[46619]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:50:58.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:57 smithi160 ceph-mon[46962]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:50:58.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:57 smithi160 ceph-mon[46962]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:50:58.385 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T21:50:59.192 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:58 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:59.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:59.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:50:59.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T21:50:59.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:58 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 -- 172.21.15.64:0/3551416771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360106db0 msgr2=0x7f53601091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 --2- 172.21.15.64:0/3551416771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360106db0 0x7f53601091a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f535800b0a0 tx=0x7f535800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 -- 172.21.15.64:0/3551416771 shutdown_connections 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 --2- 172.21.15.64:0/3551416771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360106db0 0x7f53601091a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 --2- 172.21.15.64:0/3551416771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360101b50 0x7f5360103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 --2- 172.21.15.64:0/3551416771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 0x7f5360106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 -- 172.21.15.64:0/3551416771 >> 172.21.15.64:0/3551416771 conn(0x7f53600fb4a0 msgr2=0x7f53600fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:50:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 -- 172.21.15.64:0/3551416771 shutdown_connections 2024-05-08T21:50:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.372+0000 7f5367605700 1 -- 172.21.15.64:0/3551416771 wait complete. 2024-05-08T21:50:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 Processor -- start 2024-05-08T21:50:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 -- start start 2024-05-08T21:50:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 0x7f5360101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 0x7f53600ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360106db0 0x7f5360100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53601007e0 con 0x7f5360101b50 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360100950 con 0x7f5360104480 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5367605700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53601a10a0 con 0x7f5360106db0 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f53653a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 0x7f5360101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f53653a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 0x7f5360101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58304/0 (socket says 172.21.15.64:58304) 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f53653a1700 1 -- 172.21.15.64:0/2600736455 learned_addr learned my addr 172.21.15.64:0/2600736455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5365ba2700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360106db0 0x7f5360100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.373+0000 7f5364ba0700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 0x7f53600ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53653a1700 1 -- 172.21.15.64:0/2600736455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360106db0 msgr2=0x7f5360100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53653a1700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360106db0 0x7f5360100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53653a1700 1 -- 172.21.15.64:0/2600736455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 msgr2=0x7f53600ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53653a1700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 0x7f53600ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53653a1700 1 -- 172.21.15.64:0/2600736455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358009d00 con 0x7f5360101b50 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f5365ba2700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360106db0 0x7f5360100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:50:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f5364ba0700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 0x7f53600ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53653a1700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 0x7f5360101460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e9a0 tx=0x7f535c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c009790 con 0x7f5360101b50 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f535c00cbd0 con 0x7f5360101b50 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53601a1330 con 0x7f5360101b50 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c018680 con 0x7f5360101b50 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.374+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53601a1880 con 0x7f5360101b50 2024-05-08T21:50:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.375+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53600fd020 con 0x7f5360101b50 2024-05-08T21:50:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.376+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f535c00cd40 con 0x7f5360101b50 2024-05-08T21:50:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.377+0000 7f53567fc700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f534c0a11a0 0x7f534c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:50:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.377+0000 7f5364ba0700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f534c0a11a0 0x7f534c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:50:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.377+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7f535c014070 con 0x7f5360101b50 2024-05-08T21:50:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.378+0000 7f5364ba0700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f534c0a11a0 0x7f534c0a3660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53500099d0 tx=0x7f5350008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:50:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.381+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f535c096580 con 0x7f5360101b50 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:50:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:50:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:50:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:50:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:50:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T21:50:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:58 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:50:59.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:50:59.594+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f536002ce10 con 0x7f534c0a11a0 2024-05-08T21:51:00.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 21:50:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T21:50:59.734+0000 7fc596b9d700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T21:51:00.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 21:50:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T21:50:59.744+0000 7fc58f18b700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:51:00.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi186 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi186 to 8557M 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] boot 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T21:51:00.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:50:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:51:00.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:51:00.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi186 2024-05-08T21:51:00.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi186 to 8557M 2024-05-08T21:51:00.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:51:00.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:00.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:00.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:00.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' 2024-05-08T21:51:00.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] boot 2024-05-08T21:51:00.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T21:51:00.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:50:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi186 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi186 to 8557M 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: from='osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288]' entity='osd.5' 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: osd.5 [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] boot 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T21:51:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:50:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:51:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:00 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:51:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:00 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:51:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:00 smithi160 ceph-mon[46962]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:00 smithi160 ceph-mon[46962]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T21:51:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:00 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:51:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:00 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:51:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:00 smithi186 ceph-mon[46619]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:00 smithi186 ceph-mon[46619]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T21:51:01.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:00 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:51:01.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:00 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:51:01.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:00 smithi064 ceph-mon[42500]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:01.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:00 smithi064 ceph-mon[42500]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T21:51:02.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:01 smithi160 ceph-mon[46962]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:01 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3135795030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e46cfa92-0895-463b-92e8-a8ab73a0d1e6"}]: dispatch 2024-05-08T21:51:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:01 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3135795030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e46cfa92-0895-463b-92e8-a8ab73a0d1e6"}]': finished 2024-05-08T21:51:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:01 smithi160 ceph-mon[46962]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T21:51:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:01 smithi186 ceph-mon[46619]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:01 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3135795030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e46cfa92-0895-463b-92e8-a8ab73a0d1e6"}]: dispatch 2024-05-08T21:51:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:01 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3135795030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e46cfa92-0895-463b-92e8-a8ab73a0d1e6"}]': finished 2024-05-08T21:51:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:01 smithi186 ceph-mon[46619]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T21:51:02.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:02.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:01 smithi064 ceph-mon[42500]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:02.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:01 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3135795030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e46cfa92-0895-463b-92e8-a8ab73a0d1e6"}]: dispatch 2024-05-08T21:51:02.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:01 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3135795030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e46cfa92-0895-463b-92e8-a8ab73a0d1e6"}]': finished 2024-05-08T21:51:02.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:01 smithi064 ceph-mon[42500]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T21:51:02.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:03.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/517457404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:03.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:02 smithi160 ceph-mon[46962]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T21:51:03.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:02 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/517457404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:02 smithi186 ceph-mon[46619]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T21:51:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:03.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/517457404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:03.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:02 smithi064 ceph-mon[42500]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T21:51:03.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:04.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:03 smithi160 ceph-mon[46962]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:04.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:03 smithi160 ceph-mon[46962]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T21:51:04.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:03 smithi186 ceph-mon[46619]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:03 smithi186 ceph-mon[46619]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T21:51:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:03 smithi064 ceph-mon[42500]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:03 smithi064 ceph-mon[42500]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T21:51:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:06.253 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:05 smithi064 ceph-mon[42500]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 571 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:06.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:05 smithi160 ceph-mon[46962]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 571 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:06.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:05 smithi186 ceph-mon[46619]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 571 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:08.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:07 smithi160 ceph-mon[46962]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T21:51:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:08.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:07 smithi064 ceph-mon[42500]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:08.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T21:51:08.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:08.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:07 smithi186 ceph-mon[46619]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:08.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T21:51:08.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:09.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:08 smithi064 ceph-mon[42500]: Deploying daemon osd.6 on smithi064 2024-05-08T21:51:09.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:08 smithi160 ceph-mon[46962]: Deploying daemon osd.6 on smithi064 2024-05-08T21:51:09.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:08 smithi186 ceph-mon[46619]: Deploying daemon osd.6 on smithi064 2024-05-08T21:51:10.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:09 smithi064 ceph-mon[42500]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-08T21:51:10.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:09 smithi160 ceph-mon[46962]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-08T21:51:10.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:09 smithi186 ceph-mon[46619]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-08T21:51:11.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:11 smithi064 ceph-mon[42500]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:51:11.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:11 smithi160 ceph-mon[46962]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:51:11.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:11 smithi186 ceph-mon[46619]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:51:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:12.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:12.898 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:12.898 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:12.898 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:13.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:13.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:13.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:13.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.788+0000 7f53567fc700 1 -- 172.21.15.64:0/2600736455 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f536002ce10 con 0x7f534c0a11a0 2024-05-08T21:51:13.789 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2024-05-08T21:51:13.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f534c0a11a0 msgr2=0x7f534c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f534c0a11a0 0x7f534c0a3660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53500099d0 tx=0x7f5350008040 comp rx=0 tx=0).stop 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 msgr2=0x7f5360101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 0x7f5360101460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e9a0 tx=0x7f535c00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 shutdown_connections 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f534c0a11a0 0x7f534c0a3660 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5360106db0 0x7f5360100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360101b50 0x7f5360101460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 --2- 172.21.15.64:0/2600736455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5360104480 0x7f53600ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 >> 172.21.15.64:0/2600736455 conn(0x7f53600fb4a0 msgr2=0x7f5360102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 shutdown_connections 2024-05-08T21:51:13.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:13.790+0000 7f5367605700 1 -- 172.21.15.64:0/2600736455 wait complete. 2024-05-08T21:51:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:13 smithi064 ceph-mon[42500]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T21:51:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:13 smithi160 ceph-mon[46962]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T21:51:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:14.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:13 smithi186 ceph-mon[46619]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T21:51:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:14.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.236 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T21:51:14.238 INFO:tasks.cephadm:Deploying osd.7 on smithi160 with /dev/vg_nvme/lv_2... 2024-05-08T21:51:14.239 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T21:51:14.271 INFO:journalctl@ceph.osd.6.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:51:14.471 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:51:14.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.940 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.940 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:14 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:14.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:14.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:14 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:14 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:15.800 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T21:51:15.800 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T21:51:15.800 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T21:51:15.800 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T21:51:15.800 INFO:teuthology.orchestra.run.smithi160.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120663 s, 869 MB/s 2024-05-08T21:51:15.801 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 2024-05-08T21:51:15.801 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T21:51:15.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:15 smithi064 ceph-mon[42500]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-08T21:51:15.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:15 smithi064 ceph-mon[42500]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:51:15.964 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 21:51:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T21:51:15.562+0000 7f044d1c87c0 -1 osd.6 0 log_to_monitors true 2024-05-08T21:51:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:15 smithi160 ceph-mon[46962]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-08T21:51:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:15 smithi160 ceph-mon[46962]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:51:16.140 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-05-08T21:51:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:15 smithi186 ceph-mon[46619]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-08T21:51:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:15 smithi186 ceph-mon[46619]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:51:16.360 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:51:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T21:51:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T21:51:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:51:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:17.198 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:17.199 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.213 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 21:51:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T21:51:16.795+0000 7f044202b700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T21:51:17.213 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 21:51:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T21:51:16.808+0000 7f043e621700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:51:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T21:51:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T21:51:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:51:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:17.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:17.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.377+0000 7ff6deb07700 1 -- 172.21.15.160:0/2623686527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 msgr2=0x7ff6d8109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:17.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.377+0000 7ff6deb07700 1 --2- 172.21.15.160:0/2623686527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d8109450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b0d0 tx=0x7ff6d000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:51:17.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- 172.21.15.160:0/2623686527 shutdown_connections 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 --2- 172.21.15.160:0/2623686527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d8109450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 --2- 172.21.15.160:0/2623686527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6d8101000 0x7ff6d8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 --2- 172.21.15.160:0/2623686527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6d8102200 0x7ff6d8102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- 172.21.15.160:0/2623686527 >> 172.21.15.160:0/2623686527 conn(0x7ff6d80fc5c0 msgr2=0x7ff6d80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- 172.21.15.160:0/2623686527 shutdown_connections 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- 172.21.15.160:0/2623686527 wait complete. 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 Processor -- start 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- start start 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6d8101000 0x7ff6d8074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6d8102200 0x7ff6d8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8074070 con 0x7ff6d8102200 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80741e0 con 0x7ff6d8102ba0 2024-05-08T21:51:17.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6deb07700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81a10c0 con 0x7ff6d8101000 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6dd0a4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6dd0a4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d80738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49144/0 (socket says 172.21.15.160:49144) 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.378+0000 7ff6dd0a4700 1 -- 172.21.15.160:0/35539831 learned_addr learned my addr 172.21.15.160:0/35539831 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dc8a3700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6d8101000 0x7ff6d8074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6d7fff700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6d8102200 0x7ff6d8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dd0a4700 1 -- 172.21.15.160:0/35539831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6d8101000 msgr2=0x7ff6d8074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dd0a4700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6d8101000 0x7ff6d8074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dd0a4700 1 -- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6d8102200 msgr2=0x7ff6d8073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dd0a4700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6d8102200 0x7ff6d8073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dd0a4700 1 -- 172.21.15.160:0/35539831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0009d00 con 0x7ff6d8102ba0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6dd0a4700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d80738d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007bd0 tx=0x7ff6d0009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000e040 con 0x7ff6d8102ba0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d81a12f0 con 0x7ff6d8102ba0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d001dd50 con 0x7ff6d8102ba0 2024-05-08T21:51:17.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001b450 con 0x7ff6d8102ba0 2024-05-08T21:51:17.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.379+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d81a17b0 con 0x7ff6d8102ba0 2024-05-08T21:51:17.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.380+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8108490 con 0x7ff6d8102ba0 2024-05-08T21:51:17.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.381+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff6d001d890 con 0x7ff6d8102ba0 2024-05-08T21:51:17.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.382+0000 7ff6d5ffb700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff6c80a1120 0x7ff6c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:17.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.382+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4685+0+0 (secure 0 0 0) 0x7ff6d00ce160 con 0x7ff6d8102ba0 2024-05-08T21:51:17.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.382+0000 7ff6dc8a3700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff6c80a1120 0x7ff6c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:17.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.383+0000 7ff6dc8a3700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff6c80a1120 0x7ff6c80a35e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8102060 tx=0x7ff6cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:51:17.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.386+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff6d00987b0 con 0x7ff6d8102ba0 2024-05-08T21:51:17.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:17.585+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6d802ce10 con 0x7ff6c80a1120 2024-05-08T21:51:18.080 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:51:18.080 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi064 2024-05-08T21:51:18.080 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi064 to 5704M 2024-05-08T21:51:18.081 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:51:18.081 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T21:51:18.081 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.081 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.081 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:18.081 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:18.082 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:18.082 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:51:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi064 2024-05-08T21:51:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi064 to 5704M 2024-05-08T21:51:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:51:18.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T21:51:18.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:18.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:18.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:18.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:51:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi064 2024-05-08T21:51:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi064 to 5704M 2024-05-08T21:51:18.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:51:18.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T21:51:18.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:18.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:18.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:18.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:18.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:19.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:18 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:51:19.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:18 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:51:19.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:18 smithi160 ceph-mon[46962]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:19.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:18 smithi160 ceph-mon[46962]: osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] boot 2024-05-08T21:51:19.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:18 smithi160 ceph-mon[46962]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T21:51:19.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:18 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:18 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:51:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:18 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:51:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:18 smithi186 ceph-mon[46619]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:18 smithi186 ceph-mon[46619]: osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] boot 2024-05-08T21:51:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:18 smithi186 ceph-mon[46619]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T21:51:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:18 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:18 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:51:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:18 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:51:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:18 smithi064 ceph-mon[42500]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:18 smithi064 ceph-mon[42500]: osd.6 [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] boot 2024-05-08T21:51:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:18 smithi064 ceph-mon[42500]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T21:51:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:18 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:51:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T21:51:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/4079032181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]: dispatch 2024-05-08T21:51:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]: dispatch 2024-05-08T21:51:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]': finished 2024-05-08T21:51:20.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T21:51:20.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/4079032181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]: dispatch 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]: dispatch 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]': finished 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T21:51:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:51:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T21:51:20.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/4079032181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]: dispatch 2024-05-08T21:51:20.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]: dispatch 2024-05-08T21:51:20.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc1fffb-253b-47fb-99de-85b4c9cfa6e3"}]': finished 2024-05-08T21:51:20.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T21:51:20.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:21.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:20 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/1576528259' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:20 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/1576528259' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:21.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:20 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/1576528259' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:22.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:21 smithi186 ceph-mon[46619]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:22.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:21 smithi064 ceph-mon[42500]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:22.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:21 smithi160 ceph-mon[46962]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:23 smithi186 ceph-mon[46619]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:23 smithi064 ceph-mon[42500]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:24.245 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:23 smithi160 ceph-mon[46962]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:25 smithi160 ceph-mon[46962]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:26.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T21:51:26.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:25 smithi186 ceph-mon[46619]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T21:51:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:26.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:25 smithi064 ceph-mon[42500]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:26.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T21:51:26.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:27.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:26 smithi186 ceph-mon[46619]: Deploying daemon osd.7 on smithi160 2024-05-08T21:51:27.188 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:26 smithi160 ceph-mon[46962]: Deploying daemon osd.7 on smithi160 2024-05-08T21:51:27.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:26 smithi064 ceph-mon[42500]: Deploying daemon osd.7 on smithi160 2024-05-08T21:51:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:27 smithi186 ceph-mon[46619]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:28.190 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:27 smithi160 ceph-mon[46962]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:28.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:27 smithi064 ceph-mon[42500]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:30.099 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:29 smithi160 ceph-mon[46962]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:30.100 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:30.100 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:30.100 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:29 smithi186 ceph-mon[46619]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:30.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:29 smithi064 ceph-mon[42500]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:31.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.637+0000 7ff6d5ffb700 1 -- 172.21.15.160:0/35539831 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff6d802ce10 con 0x7ff6c80a1120 2024-05-08T21:51:31.639 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 7 on host 'smithi160' 2024-05-08T21:51:31.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff6c80a1120 msgr2=0x7ff6c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:31.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff6c80a1120 0x7ff6c80a35e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8102060 tx=0x7ff6cc00b410 comp rx=0 tx=0).stop 2024-05-08T21:51:31.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 msgr2=0x7ff6d80738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:31.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d80738d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007bd0 tx=0x7ff6d0009440 comp rx=0 tx=0).stop 2024-05-08T21:51:31.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 shutdown_connections 2024-05-08T21:51:31.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff6c80a1120 0x7ff6c80a35e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:31.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6d8102ba0 0x7ff6d80738d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:31.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6d8101000 0x7ff6d8074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:31.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 --2- 172.21.15.160:0/35539831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6d8102200 0x7ff6d8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:31.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 >> 172.21.15.160:0/35539831 conn(0x7ff6d80fc5c0 msgr2=0x7ff6d81055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:51:31.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 shutdown_connections 2024-05-08T21:51:31.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:51:31.640+0000 7ff6deb07700 1 -- 172.21.15.160:0/35539831 wait complete. 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:31.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:32.021 DEBUG:teuthology.orchestra.run.smithi160:osd.7> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T21:51:32.024 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-05-08T21:51:32.024 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T21:51:32.057 INFO:journalctl@ceph.osd.7.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:51:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:32.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:32.259 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:51:33.552 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T21:51:33.552 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T21:51:33.552 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-08T21:51:33.552 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-08T21:51:33.552 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137311 s, 764 MB/s 2024-05-08T21:51:33.552 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-08T21:51:33.795 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 21:51:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T21:51:33.455+0000 7f5bb94187c0 -1 osd.7 0 log_to_monitors true 2024-05-08T21:51:33.934 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-05-08T21:51:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:33 smithi186 ceph-mon[46619]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:33 smithi186 ceph-mon[46619]: from='osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:51:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:33 smithi186 ceph-mon[46619]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:51:34.152 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:51:34.173 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:33 smithi160 ceph-mon[46962]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:34.173 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:33 smithi160 ceph-mon[46962]: from='osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:51:34.173 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:33 smithi160 ceph-mon[46962]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:51:34.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:33 smithi064 ceph-mon[42500]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:34.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:33 smithi064 ceph-mon[42500]: from='osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:51:34.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:33 smithi064 ceph-mon[42500]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 -- 172.21.15.186:0/2169509192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100260 msgr2=0x7f19381006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 --2- 172.21.15.186:0/2169509192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100260 0x7f19381006e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1928009990 tx=0x7f1928009ca0 comp rx=0 tx=0).stop 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 -- 172.21.15.186:0/2169509192 shutdown_connections 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 --2- 172.21.15.186:0/2169509192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19380ff900 0x7f19380ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 --2- 172.21.15.186:0/2169509192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1938100c20 0x7f19381010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 --2- 172.21.15.186:0/2169509192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100260 0x7f19381006e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 -- 172.21.15.186:0/2169509192 >> 172.21.15.186:0/2169509192 conn(0x7f19380fb4a0 msgr2=0x7f19380fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 -- 172.21.15.186:0/2169509192 shutdown_connections 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.114+0000 7f193f3f9700 1 -- 172.21.15.186:0/2169509192 wait complete. 2024-05-08T21:51:35.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 Processor -- start 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 -- start start 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19380ff900 0x7f1938199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1938100260 0x7f19381996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 0x7f19381a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19381a0cd0 con 0x7f1938100260 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19381a0e10 con 0x7f19380ff900 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193f3f9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19381a0fa0 con 0x7f1938100c20 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193d996700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 0x7f19381a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193d195700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19380ff900 0x7f1938199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193d996700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 0x7f19381a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57638/0 (socket says 172.21.15.186:57638) 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193d996700 1 -- 172.21.15.186:0/4083886425 learned_addr learned my addr 172.21.15.186:0/4083886425 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.115+0000 7f193c994700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1938100260 0x7f19381996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:35.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193d996700 1 -- 172.21.15.186:0/4083886425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19380ff900 msgr2=0x7f1938199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193d996700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19380ff900 0x7f1938199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193d996700 1 -- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1938100260 msgr2=0x7f19381996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193d996700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1938100260 0x7f19381996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193d996700 1 -- 172.21.15.186:0/4083886425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280095f0 con 0x7f1938100c20 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193d996700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 0x7f19381a0700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f193000add0 tx=0x7f193000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000f070 con 0x7f1938100c20 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19381a1280 con 0x7f1938100c20 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1930007b30 con 0x7f1938100c20 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930005280 con 0x7f1938100c20 2024-05-08T21:51:35.119 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.116+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19381a1780 con 0x7f1938100c20 2024-05-08T21:51:35.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.117+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193810a6e0 con 0x7f1938100c20 2024-05-08T21:51:35.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.118+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f193000c890 con 0x7f1938100c20 2024-05-08T21:51:35.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.118+0000 7f192e7fc700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19240a1120 0x7f19240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:35.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.119+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f19300cb7a0 con 0x7f1938100c20 2024-05-08T21:51:35.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.119+0000 7f193d195700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19240a1120 0x7f19240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:35.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.119+0000 7f193d195700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19240a1120 0x7f19240a35e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1934005950 tx=0x7f193400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:51:35.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.122+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1930095c40 con 0x7f1938100c20 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:35.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:35.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:35.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:35.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 21:51:34 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T21:51:34.865+0000 7f5bae27b700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T21:51:35.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 21:51:34 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T21:51:34.876+0000 7f5baa871700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:51:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T21:51:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T21:51:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:51:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:51:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:35.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:35.322 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:35.322+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f19380613e0 con 0x7f19240a1120 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi160 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi160 to 5704M 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:35.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi160 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi160 to 5704M 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:36.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:36.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi160 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi160 to 5704M 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:36.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:37.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:36 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:51:37.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:36 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:51:37.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:36 smithi186 ceph-mon[46619]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:37.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:36 smithi186 ceph-mon[46619]: osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] boot 2024-05-08T21:51:37.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:36 smithi186 ceph-mon[46619]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T21:51:37.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:36 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:37.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:36 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:51:37.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:36 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:51:37.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:36 smithi064 ceph-mon[42500]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:37.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:36 smithi064 ceph-mon[42500]: osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] boot 2024-05-08T21:51:37.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:36 smithi064 ceph-mon[42500]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T21:51:37.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:36 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:37.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:36 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:51:37.298 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:36 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:51:37.298 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:36 smithi160 ceph-mon[46962]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:37.298 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:36 smithi160 ceph-mon[46962]: osd.7 [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] boot 2024-05-08T21:51:37.298 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:36 smithi160 ceph-mon[46962]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T21:51:37.298 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:36 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: pgmap v136: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/646839157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]: dispatch 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]: dispatch 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]': finished 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T21:51:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:38.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/469575253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:38.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T21:51:38.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:38.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: pgmap v136: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/646839157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]: dispatch 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]: dispatch 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]': finished 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/469575253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T21:51:38.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: pgmap v136: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T21:51:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/646839157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]: dispatch 2024-05-08T21:51:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]: dispatch 2024-05-08T21:51:38.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "316f22c0-33f7-44c2-ae4f-0ed122d52d5f"}]': finished 2024-05-08T21:51:38.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T21:51:38.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:38.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/469575253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:38.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T21:51:38.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:40.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:39 smithi160 ceph-mon[46962]: pgmap v140: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:40.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:39 smithi160 ceph-mon[46962]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T21:51:40.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:39 smithi186 ceph-mon[46619]: pgmap v140: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:39 smithi186 ceph-mon[46619]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T21:51:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:39 smithi064 ceph-mon[42500]: pgmap v140: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:39 smithi064 ceph-mon[42500]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T21:51:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:41 smithi160 ceph-mon[46962]: pgmap v142: 1 pgs: 1 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:42.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:41 smithi186 ceph-mon[46619]: pgmap v142: 1 pgs: 1 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:42.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:41 smithi064 ceph-mon[42500]: pgmap v142: 1 pgs: 1 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:43.877 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:43 smithi186 ceph-mon[46619]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:43.877 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T21:51:43.877 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:43 smithi160 ceph-mon[46962]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T21:51:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:43 smithi064 ceph-mon[42500]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T21:51:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:44.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:44 smithi186 ceph-mon[46619]: Deploying daemon osd.8 on smithi186 2024-05-08T21:51:45.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:44 smithi160 ceph-mon[46962]: Deploying daemon osd.8 on smithi186 2024-05-08T21:51:45.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:44 smithi064 ceph-mon[42500]: Deploying daemon osd.8 on smithi186 2024-05-08T21:51:46.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:45 smithi160 ceph-mon[46962]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:45 smithi186 ceph-mon[46619]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:45 smithi064 ceph-mon[42500]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:47.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:47 smithi064 ceph-mon[42500]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:47 smithi160 ceph-mon[46962]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:48.094 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:47 smithi186 ceph-mon[46619]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:48.094 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:48.094 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:48.094 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:48.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.975+0000 7f192e7fc700 1 -- 172.21.15.186:0/4083886425 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f19380613e0 con 0x7f19240a1120 2024-05-08T21:51:48.976 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-05-08T21:51:48.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19240a1120 msgr2=0x7f19240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:48.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19240a1120 0x7f19240a35e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1934005950 tx=0x7f193400a300 comp rx=0 tx=0).stop 2024-05-08T21:51:48.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 msgr2=0x7f19381a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:48.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 0x7f19381a0700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f193000add0 tx=0x7f193000c330 comp rx=0 tx=0).stop 2024-05-08T21:51:48.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 shutdown_connections 2024-05-08T21:51:48.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19240a1120 0x7f19240a35e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:48.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19380ff900 0x7f1938199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:48.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1938100c20 0x7f19381a0700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:48.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 --2- 172.21.15.186:0/4083886425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1938100260 0x7f19381996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:48.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 >> 172.21.15.186:0/4083886425 conn(0x7f19380fb4a0 msgr2=0x7f19380fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:51:48.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 shutdown_connections 2024-05-08T21:51:48.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:51:48.976+0000 7f193f3f9700 1 -- 172.21.15.186:0/4083886425 wait complete. 2024-05-08T21:51:49.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.127 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:49.127 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:49.127 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:49.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:49.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:49.352 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T21:51:49.354 INFO:tasks.cephadm:Deploying osd.9 on smithi064 with /dev/vg_nvme/lv_1... 2024-05-08T21:51:49.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T21:51:49.401 INFO:journalctl@ceph.osd.8.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:51:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:51:50.032 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:49 smithi064 ceph-mon[42500]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:50.032 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:50.032 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:50.032 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:49 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:50.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:49 smithi186 ceph-mon[46619]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:50.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:50.126 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:50.127 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:49 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:50.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:49 smithi160 ceph-mon[46962]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:50.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:50.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:50.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:49 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:51:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T21:51:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T21:51:50.909 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-08T21:51:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-08T21:51:50.909 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112416 s, 933 MB/s 2024-05-08T21:51:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-08T21:51:50.963 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 21:51:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T21:51:50.690+0000 7f0e8aac37c0 -1 osd.8 0 log_to_monitors true 2024-05-08T21:51:51.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:50 smithi064 ceph-mon[42500]: from='osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:51:51.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:50 smithi064 ceph-mon[42500]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:51:51.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:50 smithi160 ceph-mon[46962]: from='osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:51:51.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:50 smithi160 ceph-mon[46962]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:51:51.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-05-08T21:51:51.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:50 smithi186 ceph-mon[46619]: from='osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:51:51.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:50 smithi186 ceph-mon[46619]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:51:51.531 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:52.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:51 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:52.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:52.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:51 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.391 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 21:51:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T21:51:51.971+0000 7f0e7f926700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T21:51:52.391 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 21:51:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T21:51:51.983+0000 7f0e7bf1c700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:51:52.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:52.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:51:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:51 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.505+0000 7fb150b1d700 1 -- 172.21.15.64:0/1120614049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c101490 msgr2=0x7fb14c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.505+0000 7fb150b1d700 1 --2- 172.21.15.64:0/1120614049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c101490 0x7fb14c1018b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0098f0 tx=0x7fb13c009c00 comp rx=0 tx=0).stop 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 -- 172.21.15.64:0/1120614049 shutdown_connections 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 --2- 172.21.15.64:0/1120614049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb14c100b70 0x7fb14c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 --2- 172.21.15.64:0/1120614049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c101490 0x7fb14c1018b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 --2- 172.21.15.64:0/1120614049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 0x7fb14c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:52.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 -- 172.21.15.64:0/1120614049 >> 172.21.15.64:0/1120614049 conn(0x7fb14c0fac40 msgr2=0x7fb14c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 -- 172.21.15.64:0/1120614049 shutdown_connections 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 -- 172.21.15.64:0/1120614049 wait complete. 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 Processor -- start 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.506+0000 7fb150b1d700 1 -- start start 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 0x7fb14c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 0x7fb14c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb14c101490 0x7fb14c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c19ea40 con 0x7fb14c100b70 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c19ebb0 con 0x7fb14c1001d0 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c19ed70 con 0x7fb14c101490 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 0x7fb14c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 0x7fb14c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34916/0 (socket says 172.21.15.64:34916) 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 -- 172.21.15.64:0/2184621844 learned_addr learned my addr 172.21.15.64:0/2184621844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:51:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb14ad9d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb14c101490 0x7fb14c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb14a59c700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 0x7fb14c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 -- 172.21.15.64:0/2184621844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb14c101490 msgr2=0x7fb14c19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb14c101490 0x7fb14c19e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 -- 172.21.15.64:0/2184621844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 msgr2=0x7fb14c0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 0x7fb14c0ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 -- 172.21.15.64:0/2184621844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009520 con 0x7fb14c100b70 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb14a59c700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 0x7fb14c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb149d9b700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 0x7fb14c19bcb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9c0 tx=0x7fb14000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14000cbd0 con 0x7fb14c100b70 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb14000eeb0 con 0x7fb14c100b70 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c19eff0 con 0x7fb14c100b70 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.507+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140005580 con 0x7fb14c100b70 2024-05-08T21:51:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.508+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c19f4c0 con 0x7fb14c100b70 2024-05-08T21:51:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.508+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c107500 con 0x7fb14c100b70 2024-05-08T21:51:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.510+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb140004750 con 0x7fb14c100b70 2024-05-08T21:51:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.510+0000 7fb13b7fe700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb1340a1120 0x7fb1340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:51:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.510+0000 7fb14a59c700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb1340a1120 0x7fb1340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:51:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.511+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5549+0+0 (secure 0 0 0) 0x7fb140014070 con 0x7fb14c100b70 2024-05-08T21:51:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.511+0000 7fb14a59c700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb1340a1120 0x7fb1340a35e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0098c0 tx=0x7fb13c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:51:52.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.514+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb140096710 con 0x7fb14c100b70 2024-05-08T21:51:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:51:52.730+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb14c02ce40 con 0x7fb1340a1120 2024-05-08T21:51:53.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi186 2024-05-08T21:51:53.384 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi186 to 5704M 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] boot 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T21:51:53.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:52 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi186 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi186 to 5704M 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:53.386 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:53.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] boot 2024-05-08T21:51:53.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T21:51:53.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:52 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi186 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi186 to 5704M 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:51:53.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:51:53.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:51:53.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: osd.8 [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] boot 2024-05-08T21:51:53.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T21:51:53.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:52 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:51:54.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:53 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:51:54.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:53 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:51:54.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:53 smithi160 ceph-mon[46962]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:54.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:53 smithi160 ceph-mon[46962]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:54.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:53 smithi160 ceph-mon[46962]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T21:51:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:53 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:51:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:53 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:51:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:53 smithi186 ceph-mon[46619]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:53 smithi186 ceph-mon[46619]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:53 smithi186 ceph-mon[46619]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T21:51:54.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:53 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:51:54.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:53 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:51:54.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:53 smithi064 ceph-mon[42500]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:51:54.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:53 smithi064 ceph-mon[42500]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:51:54.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:53 smithi064 ceph-mon[42500]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T21:51:55.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:54 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2828773177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bbe8d1f-ece5-435b-89db-1abeefdb4320"}]: dispatch 2024-05-08T21:51:55.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:54 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2828773177' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bbe8d1f-ece5-435b-89db-1abeefdb4320"}]': finished 2024-05-08T21:51:55.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:54 smithi160 ceph-mon[46962]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T21:51:55.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:54 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:55.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:54 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2828773177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bbe8d1f-ece5-435b-89db-1abeefdb4320"}]: dispatch 2024-05-08T21:51:55.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:54 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2828773177' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bbe8d1f-ece5-435b-89db-1abeefdb4320"}]': finished 2024-05-08T21:51:55.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:54 smithi186 ceph-mon[46619]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T21:51:55.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:54 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:55.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:54 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2828773177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bbe8d1f-ece5-435b-89db-1abeefdb4320"}]: dispatch 2024-05-08T21:51:55.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:54 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2828773177' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bbe8d1f-ece5-435b-89db-1abeefdb4320"}]': finished 2024-05-08T21:51:55.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:54 smithi064 ceph-mon[42500]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T21:51:55.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:54 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:56.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:55 smithi160 ceph-mon[46962]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:51:56.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:55 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1337324861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:55 smithi186 ceph-mon[46619]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:51:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:55 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1337324861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:56.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:55 smithi064 ceph-mon[42500]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:51:56.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:55 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1337324861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:51:57.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:56 smithi160 ceph-mon[46962]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T21:51:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:56 smithi160 ceph-mon[46962]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T21:51:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:56 smithi186 ceph-mon[46619]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T21:51:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:56 smithi186 ceph-mon[46619]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T21:51:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:57.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:56 smithi064 ceph-mon[42500]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T21:51:57.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:57.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:56 smithi064 ceph-mon[42500]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T21:51:57.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:51:58.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:51:57 smithi160 ceph-mon[46962]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T21:51:58.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:51:57 smithi186 ceph-mon[46619]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T21:51:58.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:51:57 smithi064 ceph-mon[42500]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T21:52:00.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:00 smithi160 ceph-mon[46962]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-05-08T21:52:00.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:00 smithi186 ceph-mon[46619]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-05-08T21:52:00.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:00 smithi064 ceph-mon[42500]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-05-08T21:52:01.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T21:52:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:01.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T21:52:01.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T21:52:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:02.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:02 smithi160 ceph-mon[46962]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T21:52:02.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:02 smithi160 ceph-mon[46962]: Deploying daemon osd.9 on smithi064 2024-05-08T21:52:02.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:02 smithi064 ceph-mon[42500]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T21:52:02.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:02 smithi064 ceph-mon[42500]: Deploying daemon osd.9 on smithi064 2024-05-08T21:52:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:02 smithi186 ceph-mon[46619]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T21:52:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:02 smithi186 ceph-mon[46619]: Deploying daemon osd.9 on smithi064 2024-05-08T21:52:03.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:03 smithi160 ceph-mon[46962]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T21:52:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:03 smithi186 ceph-mon[46619]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T21:52:03.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:03 smithi064 ceph-mon[42500]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T21:52:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:05 smithi186 ceph-mon[46619]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:05.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:05 smithi064 ceph-mon[42500]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:05.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:05.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:05.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:05.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:05 smithi160 ceph-mon[46962]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:05.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:05.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:05.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.036 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 9 on host 'smithi064' 2024-05-08T21:52:07.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.035+0000 7fb13b7fe700 1 -- 172.21.15.64:0/2184621844 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb14c02ce40 con 0x7fb1340a1120 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb1340a1120 msgr2=0x7fb1340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb1340a1120 0x7fb1340a35e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0098c0 tx=0x7fb13c019040 comp rx=0 tx=0).stop 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 msgr2=0x7fb14c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 0x7fb14c19bcb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9c0 tx=0x7fb14000ecd0 comp rx=0 tx=0).stop 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 shutdown_connections 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb1340a1120 0x7fb1340a35e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c1001d0 0x7fb14c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb14c101490 0x7fb14c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 --2- 172.21.15.64:0/2184621844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c100b70 0x7fb14c19bcb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:07.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 >> 172.21.15.64:0/2184621844 conn(0x7fb14c0fac40 msgr2=0x7fb14c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:07.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 shutdown_connections 2024-05-08T21:52:07.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:07.037+0000 7fb150b1d700 1 -- 172.21.15.64:0/2184621844 wait complete. 2024-05-08T21:52:07.504 DEBUG:teuthology.orchestra.run.smithi064:osd.9> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T21:52:07.506 INFO:tasks.cephadm:Deploying osd.10 on smithi160 with /dev/vg_nvme/lv_1... 2024-05-08T21:52:07.506 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T21:52:07.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:07.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:07.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:07.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:07.537 INFO:journalctl@ceph.osd.9.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:52:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:07.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:07.752 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:52:07.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:08.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 21:52:08 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T21:52:08.505+0000 7f7ed8ea77c0 -1 osd.9 0 log_to_monitors true 2024-05-08T21:52:09.150 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T21:52:09.150 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T21:52:09.150 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T21:52:09.151 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T21:52:09.151 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111711 s, 939 MB/s 2024-05-08T21:52:09.151 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T21:52:09.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:09 smithi064 ceph-mon[42500]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:09.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:09 smithi064 ceph-mon[42500]: from='osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:52:09.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:09 smithi064 ceph-mon[42500]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:52:09.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:09 smithi160 ceph-mon[46962]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:09.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:09 smithi160 ceph-mon[46962]: from='osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:52:09.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:09 smithi160 ceph-mon[46962]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:52:09.585 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-05-08T21:52:09.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:09 smithi186 ceph-mon[46619]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:09.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:09 smithi186 ceph-mon[46619]: from='osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:52:09.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:09 smithi186 ceph-mon[46619]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:52:09.810 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi064 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi064 to 4278M 2024-05-08T21:52:10.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:10.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:10.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T21:52:10.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T21:52:10.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:10.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi064 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi064 to 4278M 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:10.644 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:10.644 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.713 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 21:52:10 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T21:52:10.363+0000 7f7ecdd0a700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T21:52:10.713 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 21:52:10 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T21:52:10.380+0000 7f7eca300700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:52:10.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T21:52:10.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T21:52:10.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:10.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:52:10.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-08T21:52:10.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi064 2024-05-08T21:52:10.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi064 to 4278M 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:10.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:10.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 -- 172.21.15.160:0/1829180337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc0fff00 msgr2=0x7fd2bc100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:10.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 --2- 172.21.15.160:0/1829180337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc0fff00 0x7fd2bc100320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009990 tx=0x7fd2ac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:52:10.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 -- 172.21.15.160:0/1829180337 shutdown_connections 2024-05-08T21:52:10.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 --2- 172.21.15.160:0/1829180337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc0fff00 0x7fd2bc100320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:10.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 --2- 172.21.15.160:0/1829180337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2bc101a20 0x7fd2bc108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:10.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 --2- 172.21.15.160:0/1829180337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2bc101060 0x7fd2bc1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:10.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.825+0000 7fd2c2521700 1 -- 172.21.15.160:0/1829180337 >> 172.21.15.160:0/1829180337 conn(0x7fd2bc0fb4a0 msgr2=0x7fd2bc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:10.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.826+0000 7fd2c2521700 1 -- 172.21.15.160:0/1829180337 shutdown_connections 2024-05-08T21:52:10.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.826+0000 7fd2c2521700 1 -- 172.21.15.160:0/1829180337 wait complete. 2024-05-08T21:52:10.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.826+0000 7fd2c2521700 1 Processor -- start 2024-05-08T21:52:10.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.826+0000 7fd2c2521700 1 -- start start 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c2521700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2bc0fff00 0x7fd2bc069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c2521700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 0x7fd2bc19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c2521700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2bc101a20 0x7fd2bc19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c2521700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc19e940 con 0x7fd2bc101a20 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c2521700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc19eab0 con 0x7fd2bc101060 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c2521700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc19ec70 con 0x7fd2bc0fff00 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 0x7fd2bc19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 0x7fd2bc19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57112/0 (socket says 172.21.15.160:57112) 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 -- 172.21.15.160:0/2820139851 learned_addr learned my addr 172.21.15.160:0/2820139851 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bbfff700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2bc0fff00 0x7fd2bc069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:10.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2c0abe700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2bc101a20 0x7fd2bc19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 -- 172.21.15.160:0/2820139851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2bc0fff00 msgr2=0x7fd2bc069480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2bc0fff00 0x7fd2bc069480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 -- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2bc101a20 msgr2=0x7fd2bc19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2bc101a20 0x7fd2bc19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 -- 172.21.15.160:0/2820139851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0095f0 con 0x7fd2bc101060 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.827+0000 7fd2bb7fe700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 0x7fd2bc19bbb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000c9b0 tx=0x7fd2b000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.828+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0007b00 con 0x7fd2bc101060 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.828+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b000ce50 con 0x7fd2bc101060 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.828+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc19ef50 con 0x7fd2bc101060 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.828+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0018660 con 0x7fd2bc101060 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.828+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc19f410 con 0x7fd2bc101060 2024-05-08T21:52:10.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.829+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2bc1072b0 con 0x7fd2bc101060 2024-05-08T21:52:10.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.830+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd2b000f450 con 0x7fd2bc101060 2024-05-08T21:52:10.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.830+0000 7fd2b97fa700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2a40a11a0 0x7fd2a40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:10.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.830+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5981+0+0 (secure 0 0 0) 0x7fd2b00cc0e0 con 0x7fd2bc101060 2024-05-08T21:52:10.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.831+0000 7fd2bbfff700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2a40a11a0 0x7fd2a40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:10.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.831+0000 7fd2bbfff700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2a40a11a0 0x7fd2a40a3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00b5c0 tx=0x7fd2ac003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:10.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:10.834+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd2b00962a0 con 0x7fd2bc101060 2024-05-08T21:52:11.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:11.034+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2bc02ce10 con 0x7fd2a40a11a0 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:52:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:52:11.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:11.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:11.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:52:11.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T21:52:11.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:11.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:11.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:52:11.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:52:11.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:52:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:12.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:12 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:52:12.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:12 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:52:12.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:12 smithi186 ceph-mon[46619]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:12.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:12 smithi186 ceph-mon[46619]: osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] boot 2024-05-08T21:52:12.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:12 smithi186 ceph-mon[46619]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T21:52:12.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:12 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:12.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:12 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:52:12.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:12 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:52:12.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:12 smithi064 ceph-mon[42500]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:12.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:12 smithi064 ceph-mon[42500]: osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] boot 2024-05-08T21:52:12.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:12 smithi064 ceph-mon[42500]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T21:52:12.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:12 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:12.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:12 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:52:12.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:12 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:52:12.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:12 smithi160 ceph-mon[46962]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:12.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:12 smithi160 ceph-mon[46962]: osd.9 [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] boot 2024-05-08T21:52:12.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:12 smithi160 ceph-mon[46962]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T21:52:12.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:12 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:52:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: osdmap e63: 10 total, 10 up, 10 in 2024-05-08T21:52:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/2955149092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]: dispatch 2024-05-08T21:52:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]: dispatch 2024-05-08T21:52:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]': finished 2024-05-08T21:52:13.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T21:52:13.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: osdmap e63: 10 total, 10 up, 10 in 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/2955149092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]: dispatch 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]: dispatch 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]': finished 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T21:52:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:13.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: osdmap e63: 10 total, 10 up, 10 in 2024-05-08T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/2955149092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]: dispatch 2024-05-08T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]: dispatch 2024-05-08T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63c6380e-f30e-4139-a9a3-6250bb4c4358"}]': finished 2024-05-08T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:14.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:14 smithi186 ceph-mon[46619]: from='client.? 172.21.15.160:0/403776141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:52:14.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:14 smithi064 ceph-mon[42500]: from='client.? 172.21.15.160:0/403776141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:52:14.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:14 smithi160 ceph-mon[46962]: from='client.? 172.21.15.160:0/403776141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:52:15.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:15 smithi186 ceph-mon[46619]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:15.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:15 smithi064 ceph-mon[42500]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:15.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:15 smithi160 ceph-mon[46962]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:17.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:17 smithi186 ceph-mon[46619]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:17.711 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:17 smithi160 ceph-mon[46962]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:17.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:17 smithi064 ceph-mon[42500]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:19 smithi186 ceph-mon[46619]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T21:52:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:19 smithi186 ceph-mon[46619]: Deploying daemon osd.10 on smithi160 2024-05-08T21:52:19.646 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:19 smithi160 ceph-mon[46962]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:19.646 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T21:52:19.646 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:19.646 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:19 smithi160 ceph-mon[46962]: Deploying daemon osd.10 on smithi160 2024-05-08T21:52:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:19 smithi064 ceph-mon[42500]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T21:52:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:19.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:19 smithi064 ceph-mon[42500]: Deploying daemon osd.10 on smithi160 2024-05-08T21:52:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:21 smithi160 ceph-mon[46962]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:21 smithi186 ceph-mon[46619]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:21.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:21 smithi064 ceph-mon[42500]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:23.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:23 smithi064 ceph-mon[42500]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:23.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:23.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:23.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:23 smithi160 ceph-mon[46962]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:23 smithi186 ceph-mon[46619]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:24.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.990+0000 7fd2b97fa700 1 -- 172.21.15.160:0/2820139851 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd2bc02ce10 con 0x7fd2a40a11a0 2024-05-08T21:52:24.991 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 10 on host 'smithi160' 2024-05-08T21:52:24.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2a40a11a0 msgr2=0x7fd2a40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:24.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2a40a11a0 0x7fd2a40a3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00b5c0 tx=0x7fd2ac003790 comp rx=0 tx=0).stop 2024-05-08T21:52:24.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 msgr2=0x7fd2bc19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:24.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 0x7fd2bc19bbb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000c9b0 tx=0x7fd2b000ccc0 comp rx=0 tx=0).stop 2024-05-08T21:52:24.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 shutdown_connections 2024-05-08T21:52:24.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2a40a11a0 0x7fd2a40a3660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2bc101a20 0x7fd2bc19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2bc0fff00 0x7fd2bc069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 --2- 172.21.15.160:0/2820139851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2bc101060 0x7fd2bc19bbb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 >> 172.21.15.160:0/2820139851 conn(0x7fd2bc0fb4a0 msgr2=0x7fd2bc1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 shutdown_connections 2024-05-08T21:52:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T21:52:24.992+0000 7fd2c2521700 1 -- 172.21.15.160:0/2820139851 wait complete. 2024-05-08T21:52:25.460 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:25.460 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.460 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.460 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:25.460 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:25.460 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.461 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.461 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.461 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:25.660 DEBUG:teuthology.orchestra.run.smithi160:osd.10> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T21:52:25.662 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-05-08T21:52:25.663 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T21:52:25.692 INFO:journalctl@ceph.osd.10.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:25.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:25.897 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:52:27.039 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 21:52:26 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T21:52:26.698+0000 7f2d561337c0 -1 osd.10 0 log_to_monitors true 2024-05-08T21:52:27.279 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T21:52:27.279 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T21:52:27.279 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-08T21:52:27.279 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-08T21:52:27.279 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111896 s, 937 MB/s 2024-05-08T21:52:27.279 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-08T21:52:27.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:27 smithi160 ceph-mon[46962]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:27.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:27 smithi160 ceph-mon[46962]: from='osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:52:27.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:27 smithi160 ceph-mon[46962]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:52:27.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:27 smithi186 ceph-mon[46619]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:27.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:27 smithi186 ceph-mon[46619]: from='osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:52:27.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:27 smithi186 ceph-mon[46619]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:52:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:27 smithi064 ceph-mon[42500]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:27 smithi064 ceph-mon[42500]: from='osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:52:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:27 smithi064 ceph-mon[42500]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:52:27.716 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-05-08T21:52:27.937 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:52:28.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi160 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi160 to 4278M 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:28.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:28.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi160 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi160 to 4278M 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:28.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:28.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi160 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi160 to 4278M 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:28.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:28.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 -- 172.21.15.186:0/2588599091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 msgr2=0x7fd1b8106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/2588599091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8106850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009990 tx=0x7fd1a0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 -- 172.21.15.186:0/2588599091 shutdown_connections 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/2588599091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b8106d90 0x7fd1b8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/2588599091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8101b30 0x7fd1b8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/2588599091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8106850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 -- 172.21.15.186:0/2588599091 >> 172.21.15.186:0/2588599091 conn(0x7fd1b80fb4c0 msgr2=0x7fd1b80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 -- 172.21.15.186:0/2588599091 shutdown_connections 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 -- 172.21.15.186:0/2588599091 wait complete. 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.891+0000 7fd1bdd66700 1 Processor -- start 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 -- start start 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8101b30 0x7fd1b8196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:28.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b8106d90 0x7fd1b819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b819ea60 con 0x7fd1b8101b30 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b819eba0 con 0x7fd1b8106d90 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1bdd66700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b819ed30 con 0x7fd1b8104460 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:43780/0 (socket says 172.21.15.186:43780) 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 -- 172.21.15.186:0/607993455 learned_addr learned my addr 172.21.15.186:0/607993455 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1b7fff700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b8106d90 0x7fd1b819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 -- 172.21.15.186:0/607993455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b8106d90 msgr2=0x7fd1b819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b8106d90 0x7fd1b819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 -- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8101b30 msgr2=0x7fd1b8196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1b77fe700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8101b30 0x7fd1b8196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8101b30 0x7fd1b8196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:28.895 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.892+0000 7fd1aeffd700 1 -- 172.21.15.186:0/607993455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a00095f0 con 0x7fd1b8104460 2024-05-08T21:52:28.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.893+0000 7fd1aeffd700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8197460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b790 tx=0x7fd1a0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:28.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.893+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a001d070 con 0x7fd1b8104460 2024-05-08T21:52:28.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.893+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b819efb0 con 0x7fd1b8104460 2024-05-08T21:52:28.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.893+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1a0005030 con 0x7fd1b8104460 2024-05-08T21:52:28.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.893+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0004d00 con 0x7fd1b8104460 2024-05-08T21:52:28.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.893+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b819f310 con 0x7fd1b8104460 2024-05-08T21:52:28.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.894+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b80fd040 con 0x7fd1b8104460 2024-05-08T21:52:28.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.895+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd1a00051a0 con 0x7fd1b8104460 2024-05-08T21:52:28.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.895+0000 7fd1b57fa700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd1a40a1120 0x7fd1a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:28.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.896+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd1a00cd6d0 con 0x7fd1b8104460 2024-05-08T21:52:28.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.896+0000 7fd1b77fe700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd1a40a1120 0x7fd1a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:28.898 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.896+0000 7fd1b77fe700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd1a40a1120 0x7fd1a40a35e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005950 tx=0x7fd1a80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:28.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:28.899+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1a00975c0 con 0x7fd1b8104460 2024-05-08T21:52:29.100 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:29.100+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1b802ce10 con 0x7fd1a40a1120 2024-05-08T21:52:29.546 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 21:52:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T21:52:29.057+0000 7f2d4af96700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T21:52:29.546 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 21:52:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T21:52:29.072+0000 7f2d43584700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:52:29.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='osd.10 ' entity='osd.10' 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:52:29.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='osd.10 ' entity='osd.10' 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:52:29.628 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='osd.10 ' entity='osd.10' 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:52:29.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:30.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:52:30.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:52:30.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:30.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] boot 2024-05-08T21:52:30.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T21:52:30.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:30.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:30 smithi160 ceph-mon[46962]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] boot 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:30.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:30 smithi186 ceph-mon[46619]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: osd.10 [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] boot 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:52:30.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:30 smithi064 ceph-mon[42500]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:31 smithi160 ceph-mon[46962]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:31 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/1531391262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]: dispatch 2024-05-08T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:31 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]: dispatch 2024-05-08T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:31 smithi160 ceph-mon[46962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]': finished 2024-05-08T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:31 smithi160 ceph-mon[46962]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:31 smithi186 ceph-mon[46619]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:31 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/1531391262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]: dispatch 2024-05-08T21:52:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:31 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]: dispatch 2024-05-08T21:52:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:31 smithi186 ceph-mon[46619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]': finished 2024-05-08T21:52:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:31 smithi186 ceph-mon[46619]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T21:52:31.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:31 smithi064 ceph-mon[42500]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:52:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:31 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/1531391262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]: dispatch 2024-05-08T21:52:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:31 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]: dispatch 2024-05-08T21:52:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:31 smithi064 ceph-mon[42500]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a05f3067-983c-4656-9914-05ab90f5bd9d"}]': finished 2024-05-08T21:52:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:31 smithi064 ceph-mon[42500]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T21:52:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:32.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:32 smithi160 ceph-mon[46962]: from='client.? 172.21.15.186:0/1779373111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:52:32.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:32 smithi186 ceph-mon[46619]: from='client.? 172.21.15.186:0/1779373111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:52:32.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:32 smithi064 ceph-mon[42500]: from='client.? 172.21.15.186:0/1779373111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:52:33.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:33 smithi160 ceph-mon[46962]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:33.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:33 smithi160 ceph-mon[46962]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T21:52:33.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:33.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:33 smithi186 ceph-mon[46619]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:33.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:33 smithi186 ceph-mon[46619]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T21:52:33.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:33.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:33 smithi064 ceph-mon[42500]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:33 smithi064 ceph-mon[42500]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T21:52:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:34.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:34 smithi186 ceph-mon[46619]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T21:52:34.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:34.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:34 smithi064 ceph-mon[42500]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T21:52:34.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:34.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:34 smithi160 ceph-mon[46962]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T21:52:34.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:35.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:35 smithi186 ceph-mon[46619]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:35.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:35 smithi064 ceph-mon[42500]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:35.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:35 smithi160 ceph-mon[46962]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:37.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:37 smithi186 ceph-mon[46619]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-08T21:52:37.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T21:52:37.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:37.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:37 smithi064 ceph-mon[42500]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-08T21:52:37.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T21:52:37.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:37.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:37 smithi160 ceph-mon[46962]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-08T21:52:37.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T21:52:37.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:38.585 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:38 smithi186 ceph-mon[46619]: Deploying daemon osd.11 on smithi186 2024-05-08T21:52:38.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:38 smithi064 ceph-mon[42500]: Deploying daemon osd.11 on smithi186 2024-05-08T21:52:38.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:38 smithi160 ceph-mon[46962]: Deploying daemon osd.11 on smithi186 2024-05-08T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:39 smithi186 ceph-mon[46619]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-08T21:52:39.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:39 smithi064 ceph-mon[42500]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-08T21:52:39.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:39 smithi160 ceph-mon[46962]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-08T21:52:41.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:41 smithi186 ceph-mon[46619]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:52:41.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:41.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:41.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:41 smithi064 ceph-mon[42500]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:52:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:42.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:41 smithi160 ceph-mon[46962]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:52:42.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:42.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:42.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:43.309 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-05-08T21:52:43.309 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.307+0000 7fd1b57fa700 1 -- 172.21.15.186:0/607993455 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd1b802ce10 con 0x7fd1a40a1120 2024-05-08T21:52:43.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.309+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd1a40a1120 msgr2=0x7fd1a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:43.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.309+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd1a40a1120 0x7fd1a40a35e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005950 tx=0x7fd1a80058e0 comp rx=0 tx=0).stop 2024-05-08T21:52:43.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.309+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 msgr2=0x7fd1b8197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:43.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.309+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8197460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b790 tx=0x7fd1a0004730 comp rx=0 tx=0).stop 2024-05-08T21:52:43.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 shutdown_connections 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd1a40a1120 0x7fd1a40a35e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b8106d90 0x7fd1b819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8101b30 0x7fd1b8196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 --2- 172.21.15.186:0/607993455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1b8104460 0x7fd1b8197460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 >> 172.21.15.186:0/607993455 conn(0x7fd1b80fb4c0 msgr2=0x7fd1b81025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 shutdown_connections 2024-05-08T21:52:43.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:43.310+0000 7fd1bdd66700 1 -- 172.21.15.186:0/607993455 wait complete. 2024-05-08T21:52:43.710 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T21:52:43.712 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T21:52:43.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T21:52:43.743 INFO:journalctl@ceph.osd.11.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:52:43.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:43.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.911+0000 7f7fcce4f700 1 -- 172.21.15.64:0/2023150856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 msgr2=0x7f7fc81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.911+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/2023150856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81014e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009990 tx=0x7f7fbc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 -- 172.21.15.64:0/2023150856 shutdown_connections 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/2023150856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fc80fff00 0x7f7fc8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/2023150856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8101a20 0x7f7fc8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/2023150856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81014e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 -- 172.21.15.64:0/2023150856 >> 172.21.15.64:0/2023150856 conn(0x7f7fc80fb4a0 msgr2=0x7f7fc80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:44.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 -- 172.21.15.64:0/2023150856 shutdown_connections 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 -- 172.21.15.64:0/2023150856 wait complete. 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 Processor -- start 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.912+0000 7f7fcce4f700 1 -- start start 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fc80fff00 0x7f7fc81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8101a20 0x7f7fc81a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81a0cf0 con 0x7f7fc8101060 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81a0e30 con 0x7f7fc80fff00 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81a0fc0 con 0x7f7fc8101a20 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42082/0 (socket says 172.21.15.64:42082) 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 -- 172.21.15.64:0/135200085 learned_addr learned my addr 172.21.15.64:0/135200085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc659c700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fc80fff00 0x7f7fc81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc6d9d700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8101a20 0x7f7fc81a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 -- 172.21.15.64:0/135200085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8101a20 msgr2=0x7f7fc81a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8101a20 0x7f7fc81a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 -- 172.21.15.64:0/135200085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fc80fff00 msgr2=0x7f7fc81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fc80fff00 0x7f7fc81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 -- 172.21.15.64:0/135200085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc0095f0 con 0x7f7fc8101060 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fc5d9b700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81996f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b690 tx=0x7f7fbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc01d070 con 0x7f7fc8101060 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc81a1240 con 0x7f7fc8101060 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fbc005030 con 0x7f7fc8101060 2024-05-08T21:52:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc004c60 con 0x7f7fc8101060 2024-05-08T21:52:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.913+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc81a15f0 con 0x7f7fc8101060 2024-05-08T21:52:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.914+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc804ec60 con 0x7f7fc8101060 2024-05-08T21:52:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.916+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7fbc0051a0 con 0x7f7fc8101060 2024-05-08T21:52:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.917+0000 7f7fb77fe700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7fb00a1120 0x7f7fb00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.917+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7fbc0cd930 con 0x7f7fc8101060 2024-05-08T21:52:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.917+0000 7f7fc659c700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7fb00a1120 0x7f7fb00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.917+0000 7f7fc659c700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7fb00a1120 0x7f7fb00a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005950 tx=0x7f7fb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:44.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:44.921+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7fbc097730 con 0x7f7fc8101060 2024-05-08T21:52:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.124+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7fc810b710 con 0x7f7fc8101060 2024-05-08T21:52:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.125+0000 7f7fb77fe700 1 -- 172.21.15.64:0/135200085 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7fbc09ad50 con 0x7f7fc8101060 2024-05-08T21:52:45.125 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7fb00a1120 msgr2=0x7f7fb00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7fb00a1120 0x7f7fb00a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005950 tx=0x7f7fb800b410 comp rx=0 tx=0).stop 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 msgr2=0x7f7fc81996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81996f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b690 tx=0x7f7fbc004730 comp rx=0 tx=0).stop 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 shutdown_connections 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7fb00a1120 0x7f7fb00a35e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fc80fff00 0x7f7fc81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8101a20 0x7f7fc81a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 --2- 172.21.15.64:0/135200085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fc8101060 0x7f7fc81996f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 >> 172.21.15.64:0/135200085 conn(0x7f7fc80fb4a0 msgr2=0x7f7fc80689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 shutdown_connections 2024-05-08T21:52:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:45.127+0000 7f7fcce4f700 1 -- 172.21.15.64:0/135200085 wait complete. 2024-05-08T21:52:45.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 21:52:44 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T21:52:44.799+0000 7f7f0887d7c0 -1 osd.11 0 log_to_monitors true 2024-05-08T21:52:45.505 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715205149,"num_in_osds":12,"osd_in_since":1715205150,"num_remapped_pgs":0} 2024-05-08T21:52:45.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:45 smithi064 ceph-mon[42500]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T21:52:45.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:45 smithi064 ceph-mon[42500]: from='osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:52:45.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:45 smithi064 ceph-mon[42500]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:52:45.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:45 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/135200085' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:46.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:45 smithi160 ceph-mon[46962]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T21:52:46.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:45 smithi160 ceph-mon[46962]: from='osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:52:46.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:45 smithi160 ceph-mon[46962]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:52:46.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:45 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/135200085' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:45 smithi186 ceph-mon[46619]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T21:52:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:45 smithi186 ceph-mon[46619]: from='osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:52:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:45 smithi186 ceph-mon[46619]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:52:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:45 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/135200085' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:46.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T21:52:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: Detected new or changed devices on smithi186 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:46.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi186 to 4278M 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:46.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:46 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: Detected new or changed devices on smithi186 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi186 to 4278M 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:47.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:46 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: Detected new or changed devices on smithi186 2024-05-08T21:52:47.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi186 to 4278M 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:47.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:46 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 -- 172.21.15.64:0/2316274522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b8100fe0 msgr2=0x7f38b8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 --2- 172.21.15.64:0/2316274522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b8100fe0 0x7f38b8101400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0099c0 tx=0x7f38ac009cd0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 -- 172.21.15.64:0/2316274522 shutdown_connections 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 --2- 172.21.15.64:0/2316274522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38b8102b80 0x7f38b8109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 --2- 172.21.15.64:0/2316274522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b8100fe0 0x7f38b8101400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 --2- 172.21.15.64:0/2316274522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38b81021e0 0x7f38b8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 -- 172.21.15.64:0/2316274522 >> 172.21.15.64:0/2316274522 conn(0x7f38b80fc580 msgr2=0x7f38b80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 -- 172.21.15.64:0/2316274522 shutdown_connections 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.731+0000 7f38bee60700 1 -- 172.21.15.64:0/2316274522 wait complete. 2024-05-08T21:52:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 Processor -- start 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 -- start start 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38b8100fe0 0x7f38b8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 0x7f38b8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38b8102b80 0x7f38b819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b819eaa0 con 0x7f38b81021e0 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b819ebe0 con 0x7f38b8102b80 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bee60700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b819ed70 con 0x7f38b8100fe0 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 0x7f38b8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 0x7f38b8197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42110/0 (socket says 172.21.15.64:42110) 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38b7fff700 1 -- 172.21.15.64:0/4100581528 learned_addr learned my addr 172.21.15.64:0/4100581528 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bd3fd700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38b8102b80 0x7f38b819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.732+0000 7f38bcbfc700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38b8100fe0 0x7f38b8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b7fff700 1 -- 172.21.15.64:0/4100581528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38b8100fe0 msgr2=0x7f38b8197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:47.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b7fff700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38b8100fe0 0x7f38b8197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b7fff700 1 -- 172.21.15.64:0/4100581528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38b8102b80 msgr2=0x7f38b819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b7fff700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38b8102b80 0x7f38b819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b7fff700 1 -- 172.21.15.64:0/4100581528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac0095f0 con 0x7f38b81021e0 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b7fff700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 0x7f38b8197540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f38a400e9d0 tx=0x7f38a400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4009790 con 0x7f38b81021e0 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b819eff0 con 0x7f38b81021e0 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a400ee70 con 0x7f38b81021e0 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4010ea0 con 0x7f38b81021e0 2024-05-08T21:52:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.733+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b819f4c0 con 0x7f38b81021e0 2024-05-08T21:52:47.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.734+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b81084b0 con 0x7f38b81021e0 2024-05-08T21:52:47.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.735+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f38a400cbd0 con 0x7f38b81021e0 2024-05-08T21:52:47.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.736+0000 7f38b5ffb700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f38a80a1120 0x7f38a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:47.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.736+0000 7f38bcbfc700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f38a80a1120 0x7f38a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:47.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.736+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f38a4014070 con 0x7f38b81021e0 2024-05-08T21:52:47.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.736+0000 7f38bcbfc700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f38a80a1120 0x7f38a80a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009990 tx=0x7f38ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:47.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.740+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38a4052d60 con 0x7f38b81021e0 2024-05-08T21:52:47.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 21:52:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T21:52:47.526+0000 7f7efd6e0700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T21:52:47.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 21:52:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T21:52:47.543+0000 7f7ef5cce700 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:52:47.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:47 smithi186 ceph-mon[46619]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:47.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:47 smithi186 ceph-mon[46619]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:52:47.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:47 smithi186 ceph-mon[46619]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T21:52:47.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:47.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:47 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:47.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:47 smithi186 ceph-mon[46619]: from='osd.11 ' entity='osd.11' 2024-05-08T21:52:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.943+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f38b804ec60 con 0x7f38b81021e0 2024-05-08T21:52:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.944+0000 7f38b5ffb700 1 -- 172.21.15.64:0/4100581528 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f38a40961f0 con 0x7f38b81021e0 2024-05-08T21:52:47.944 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:52:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f38a80a1120 msgr2=0x7f38a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f38a80a1120 0x7f38a80a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009990 tx=0x7f38ac000bc0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 msgr2=0x7f38b8197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 0x7f38b8197540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f38a400e9d0 tx=0x7f38a400ece0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 shutdown_connections 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f38a80a1120 0x7f38a80a35e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38b8102b80 0x7f38b819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38b8100fe0 0x7f38b8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 --2- 172.21.15.64:0/4100581528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b81021e0 0x7f38b8197540 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 >> 172.21.15.64:0/4100581528 conn(0x7f38b80fc580 msgr2=0x7f38b81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 shutdown_connections 2024-05-08T21:52:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:47.946+0000 7f38bee60700 1 -- 172.21.15.64:0/4100581528 wait complete. 2024-05-08T21:52:47.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:47 smithi064 ceph-mon[42500]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:47 smithi064 ceph-mon[42500]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:52:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:47 smithi064 ceph-mon[42500]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T21:52:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:47 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:47.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:47 smithi064 ceph-mon[42500]: from='osd.11 ' entity='osd.11' 2024-05-08T21:52:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:47 smithi160 ceph-mon[46962]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:47 smithi160 ceph-mon[46962]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-08T21:52:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:47 smithi160 ceph-mon[46962]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T21:52:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:47 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:48.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:47 smithi160 ceph-mon[46962]: from='osd.11 ' entity='osd.11' 2024-05-08T21:52:48.328 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1715205149,"num_in_osds":12,"osd_in_since":1715205150,"num_remapped_pgs":0} 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: purged_snaps scrub starts 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: purged_snaps scrub ok 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/4100581528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] boot 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T21:52:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:48 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:49.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: purged_snaps scrub starts 2024-05-08T21:52:49.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: purged_snaps scrub ok 2024-05-08T21:52:49.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:49.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/4100581528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:49.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] boot 2024-05-08T21:52:49.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T21:52:49.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:48 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: purged_snaps scrub starts 2024-05-08T21:52:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: purged_snaps scrub ok 2024-05-08T21:52:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/4100581528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: osd.11 [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] boot 2024-05-08T21:52:49.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T21:52:49.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:48 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:52:49.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T21:52:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:52:49.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:49 smithi064 ceph-mon[42500]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:50.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:49 smithi160 ceph-mon[46962]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:50.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:49 smithi186 ceph-mon[46619]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:52:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 -- 172.21.15.64:0/3854214249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e94101a20 msgr2=0x7f5e94108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 --2- 172.21.15.64:0/3854214249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e94101a20 0x7f5e94108340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00b0a0 tx=0x7f5e8c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 -- 172.21.15.64:0/3854214249 shutdown_connections 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 --2- 172.21.15.64:0/3854214249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e940fff00 0x7f5e94100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 --2- 172.21.15.64:0/3854214249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e94101a20 0x7f5e94108340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 --2- 172.21.15.64:0/3854214249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e94101060 0x7f5e941014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 -- 172.21.15.64:0/3854214249 >> 172.21.15.64:0/3854214249 conn(0x7f5e940fb4c0 msgr2=0x7f5e940fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 -- 172.21.15.64:0/3854214249 shutdown_connections 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.511+0000 7f5e9a298700 1 -- 172.21.15.64:0/3854214249 wait complete. 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 Processor -- start 2024-05-08T21:52:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 -- start start 2024-05-08T21:52:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 0x7f5e94196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e94101060 0x7f5e94197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e94101a20 0x7f5e9419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9419ea00 con 0x7f5e940fff00 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9419eb40 con 0x7f5e94101060 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.512+0000 7f5e9a298700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9419ecd0 con 0x7f5e94101a20 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 0x7f5e94196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 0x7f5e94196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42120/0 (socket says 172.21.15.64:42120) 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 -- 172.21.15.64:0/2372921843 learned_addr learned my addr 172.21.15.64:0/2372921843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e937fe700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e94101060 0x7f5e94197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e98835700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e94101a20 0x7f5e9419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 -- 172.21.15.64:0/2372921843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e94101a20 msgr2=0x7f5e9419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e94101a20 0x7f5e9419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 -- 172.21.15.64:0/2372921843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e94101060 msgr2=0x7f5e94197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e94101060 0x7f5e94197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 -- 172.21.15.64:0/2372921843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c009d00 con 0x7f5e940fff00 2024-05-08T21:52:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e93fff700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 0x7f5e94196ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c9e0 tx=0x7f5e8400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84007ac0 con 0x7f5e940fff00 2024-05-08T21:52:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e8400ce80 con 0x7f5e940fff00 2024-05-08T21:52:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9419efb0 con 0x7f5e940fff00 2024-05-08T21:52:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84018660 con 0x7f5e940fff00 2024-05-08T21:52:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.513+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9419f3c0 con 0x7f5e940fff00 2024-05-08T21:52:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.514+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9404ec60 con 0x7f5e940fff00 2024-05-08T21:52:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.515+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5e8400f450 con 0x7f5e940fff00 2024-05-08T21:52:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.516+0000 7f5e917fa700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5e7c0a1120 0x7f5e7c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.516+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e840cc370 con 0x7f5e940fff00 2024-05-08T21:52:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.516+0000 7f5e937fe700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5e7c0a1120 0x7f5e7c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.516+0000 7f5e937fe700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5e7c0a1120 0x7f5e7c0a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800ac50 tx=0x7f5e8800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:50.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.520+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5e84095fa0 con 0x7f5e940fff00 2024-05-08T21:52:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.725+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5e9410b710 con 0x7f5e940fff00 2024-05-08T21:52:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.725+0000 7f5e917fa700 1 -- 172.21.15.64:0/2372921843 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5e840995c0 con 0x7f5e940fff00 2024-05-08T21:52:50.726 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:52:50.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5e7c0a1120 msgr2=0x7f5e7c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5e7c0a1120 0x7f5e7c0a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800ac50 tx=0x7f5e8800a380 comp rx=0 tx=0).stop 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 msgr2=0x7f5e94196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 0x7f5e94196ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c9e0 tx=0x7f5e8400ccf0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 shutdown_connections 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5e7c0a1120 0x7f5e7c0a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e94101a20 0x7f5e9419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fff00 0x7f5e94196ec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 --2- 172.21.15.64:0/2372921843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e94101060 0x7f5e94197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 >> 172.21.15.64:0/2372921843 conn(0x7f5e940fb4c0 msgr2=0x7f5e940691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:50.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 shutdown_connections 2024-05-08T21:52:50.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:50.728+0000 7f5e9a298700 1 -- 172.21.15.64:0/2372921843 wait complete. 2024-05-08T21:52:51.010 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:50 smithi064 ceph-mon[42500]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T21:52:51.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:50 smithi160 ceph-mon[46962]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T21:52:51.103 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1715205168,"num_in_osds":12,"osd_in_since":1715205150,"num_remapped_pgs":0} 2024-05-08T21:52:51.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T21:52:51.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:50 smithi186 ceph-mon[46619]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T21:52:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:52:52.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:51 smithi160 ceph-mon[46962]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:52.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:51 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2372921843' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:52.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:51 smithi064 ceph-mon[42500]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:52.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:51 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2372921843' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:51 smithi186 ceph-mon[46619]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:51 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2372921843' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 -- 172.21.15.64:0/846057375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 msgr2=0x7f41e410d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 --2- 172.21.15.64:0/846057375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e410d460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009990 tx=0x7f41d4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 -- 172.21.15.64:0/846057375 shutdown_connections 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 --2- 172.21.15.64:0/846057375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e40731d0 0x7f41e40735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 --2- 172.21.15.64:0/846057375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e410d9a0 0x7f41e410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 --2- 172.21.15.64:0/846057375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e410d460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 -- 172.21.15.64:0/846057375 >> 172.21.15.64:0/846057375 conn(0x7f41e4078100 msgr2=0x7f41e4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 -- 172.21.15.64:0/846057375 shutdown_connections 2024-05-08T21:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.253+0000 7f41eb80d700 1 -- 172.21.15.64:0/846057375 wait complete. 2024-05-08T21:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 Processor -- start 2024-05-08T21:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 -- start start 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e40731d0 0x7f41e41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e410d9a0 0x7f41e419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e419e4f0 con 0x7f41e4073af0 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e419e660 con 0x7f41e410d9a0 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41eb80d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e419e820 con 0x7f41e40731d0 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e4199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42146/0 (socket says 172.21.15.64:42146) 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 -- 172.21.15.64:0/3456353166 learned_addr learned my addr 172.21.15.64:0/3456353166 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e9daa700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e410d9a0 0x7f41e419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e95a9700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e40731d0 0x7f41e41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 -- 172.21.15.64:0/3456353166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e40731d0 msgr2=0x7f41e41995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e40731d0 0x7f41e41995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 -- 172.21.15.64:0/3456353166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e410d9a0 msgr2=0x7f41e419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e410d9a0 0x7f41e419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.254+0000 7f41e8da8700 1 -- 172.21.15.64:0/3456353166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40095f0 con 0x7f41e4073af0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.255+0000 7f41e8da8700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e4199b30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098e0 tx=0x7f41d400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.255+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d401d070 con 0x7f41e4073af0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.255+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41d4022470 con 0x7f41e4073af0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.255+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4017650 con 0x7f41e4073af0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.255+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e419eaa0 con 0x7f41e4073af0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.255+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e41a6360 con 0x7f41e4073af0 2024-05-08T21:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.256+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e4108bb0 con 0x7f41e4073af0 2024-05-08T21:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.257+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f41d40225e0 con 0x7f41e4073af0 2024-05-08T21:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.258+0000 7f41da7fc700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f41d00a11a0 0x7f41d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.258+0000 7f41e95a9700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f41d00a11a0 0x7f41d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.258+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41d40cd410 con 0x7f41e4073af0 2024-05-08T21:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.258+0000 7f41e95a9700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f41d00a11a0 0x7f41d00a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41e0005950 tx=0x7f41e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:52.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.262+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f41d40970d0 con 0x7f41e4073af0 2024-05-08T21:52:52.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.465+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41e419a900 con 0x7f41e4073af0 2024-05-08T21:52:52.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.466+0000 7f41da7fc700 1 -- 172.21.15.64:0/3456353166 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f41d409a6f0 con 0x7f41e4073af0 2024-05-08T21:52:52.467 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:52:52.467 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:52:49.704785+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f41d00a11a0 msgr2=0x7f41d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f41d00a11a0 0x7f41d00a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41e0005950 tx=0x7f41e00058e0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 msgr2=0x7f41e4199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e4199b30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098e0 tx=0x7f41d400f670 comp rx=0 tx=0).stop 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 shutdown_connections 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f41d00a11a0 0x7f41d00a3660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e410d9a0 0x7f41e419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e40731d0 0x7f41e41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 --2- 172.21.15.64:0/3456353166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41e4073af0 0x7f41e4199b30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:52.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 >> 172.21.15.64:0/3456353166 conn(0x7f41e4078100 msgr2=0x7f41e40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:52.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 shutdown_connections 2024-05-08T21:52:52.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:52.468+0000 7f41eb80d700 1 -- 172.21.15.64:0/3456353166 wait complete. 2024-05-08T21:52:52.777 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:52 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3456353166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:52:52.797 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:52:52.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T21:52:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:52:53.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:52 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3456353166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:52:53.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:52 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3456353166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 -- 172.21.15.64:0/596712796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c105f00 msgr2=0x7f227c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 --2- 172.21.15.64:0/596712796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c105f00 0x7f227c1062e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2278009990 tx=0x7f2278009ca0 comp rx=0 tx=0).stop 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 -- 172.21.15.64:0/596712796 shutdown_connections 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 --2- 172.21.15.64:0/596712796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f227c100890 0x7f227c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 --2- 172.21.15.64:0/596712796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c105f00 0x7f227c1062e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 --2- 172.21.15.64:0/596712796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c0ffed0 0x7f227c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:53.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 -- 172.21.15.64:0/596712796 >> 172.21.15.64:0/596712796 conn(0x7f227c075f80 msgr2=0x7f227c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 -- 172.21.15.64:0/596712796 shutdown_connections 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.980+0000 7f22840ff700 1 -- 172.21.15.64:0/596712796 wait complete. 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 Processor -- start 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 -- start start 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f227c0ffed0 0x7f227c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 0x7f227c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c105f00 0x7f227c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19e570 con 0x7f227c100890 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19e6e0 con 0x7f227c0ffed0 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f22840ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19e8a0 con 0x7f227c105f00 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f228169a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 0x7f227c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f228169a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 0x7f227c199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60568/0 (socket says 172.21.15.64:60568) 2024-05-08T21:52:53.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.981+0000 7f228169a700 1 -- 172.21.15.64:0/3557967834 learned_addr learned my addr 172.21.15.64:0/3557967834 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f2281e9b700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f227c0ffed0 0x7f227c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228269c700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c105f00 0x7f227c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228169a700 1 -- 172.21.15.64:0/3557967834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c105f00 msgr2=0x7f227c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228169a700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c105f00 0x7f227c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228169a700 1 -- 172.21.15.64:0/3557967834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f227c0ffed0 msgr2=0x7f227c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228169a700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f227c0ffed0 0x7f227c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228169a700 1 -- 172.21.15.64:0/3557967834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780095f0 con 0x7f227c100890 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228269c700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c105f00 0x7f227c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f228169a700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 0x7f227c199c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c920 tx=0x7f226c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c007b00 con 0x7f227c100890 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f226c00ce70 con 0x7f227c100890 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c19eb80 con 0x7f227c100890 2024-05-08T21:52:53.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c0054e0 con 0x7f227c100890 2024-05-08T21:52:53.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.982+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c1a63e0 con 0x7f227c100890 2024-05-08T21:52:53.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.983+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c1015b0 con 0x7f227c100890 2024-05-08T21:52:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.984+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f226c004750 con 0x7f227c100890 2024-05-08T21:52:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.985+0000 7f2272ffd700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22680a1120 0x7f22680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.985+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f226c0cc3d0 con 0x7f227c100890 2024-05-08T21:52:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.985+0000 7f2281e9b700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22680a1120 0x7f22680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.986+0000 7f2281e9b700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22680a1120 0x7f22680a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f227800b5c0 tx=0x7f2278005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:53.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:53.989+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f226c096000 con 0x7f227c100890 2024-05-08T21:52:54.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:53 smithi160 ceph-mon[46962]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:54.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:53 smithi186 ceph-mon[46619]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:54.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.193+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f227c19a980 con 0x7f227c100890 2024-05-08T21:52:54.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.193+0000 7f2272ffd700 1 -- 172.21.15.64:0/3557967834 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f226c099620 con 0x7f227c100890 2024-05-08T21:52:54.195 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:52:54.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22680a1120 msgr2=0x7f22680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:54.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22680a1120 0x7f22680a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f227800b5c0 tx=0x7f2278005960 comp rx=0 tx=0).stop 2024-05-08T21:52:54.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 msgr2=0x7f227c199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:54.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 0x7f227c199c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c920 tx=0x7f226c00cc30 comp rx=0 tx=0).stop 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 shutdown_connections 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22680a1120 0x7f22680a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f227c0ffed0 0x7f227c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f227c105f00 0x7f227c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 --2- 172.21.15.64:0/3557967834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f227c100890 0x7f227c199c00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 >> 172.21.15.64:0/3557967834 conn(0x7f227c075f80 msgr2=0x7f227c068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 shutdown_connections 2024-05-08T21:52:54.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:54.196+0000 7f22840ff700 1 -- 172.21.15.64:0/3557967834 wait complete. 2024-05-08T21:52:54.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:53 smithi064 ceph-mon[42500]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.a on smithi064 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.d on smithi064 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.g on smithi064 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.j on smithi064 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.b on smithi160 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.e on smithi160 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.h on smithi160 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.k on smithi160 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-05-08T21:52:54.579 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-05-08T21:52:54.580 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-05-08T21:52:54.818 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:52:54.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:54 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3557967834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:52:54.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:54 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3557967834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:52:55.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:54 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3557967834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:52:55.750 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 -- 172.21.15.186:0/3962744243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101a20 msgr2=0x7f5308108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:55.750 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 --2- 172.21.15.186:0/3962744243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101a20 0x7f5308108340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f530000b0a0 tx=0x7f530000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.750 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 -- 172.21.15.186:0/3962744243 shutdown_connections 2024-05-08T21:52:55.750 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 --2- 172.21.15.186:0/3962744243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53080fff00 0x7f5308100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.750 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 --2- 172.21.15.186:0/3962744243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101a20 0x7f5308108340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 --2- 172.21.15.186:0/3962744243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5308101060 0x7f53081014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 -- 172.21.15.186:0/3962744243 >> 172.21.15.186:0/3962744243 conn(0x7f53080fb4a0 msgr2=0x7f53080fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 -- 172.21.15.186:0/3962744243 shutdown_connections 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.747+0000 7f530f2c2700 1 -- 172.21.15.186:0/3962744243 wait complete. 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 Processor -- start 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 -- start start 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53080fff00 0x7f5308069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 0x7f530819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5308101a20 0x7f530819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530819e970 con 0x7f53080fff00 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530819eae0 con 0x7f5308101a20 2024-05-08T21:52:55.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530f2c2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530819eca0 con 0x7f5308101060 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 0x7f530819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 0x7f530819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:52822/0 (socket says 172.21.15.186:52822) 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 -- 172.21.15.186:0/519072736 learned_addr learned my addr 172.21.15.186:0/519072736 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530d05e700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53080fff00 0x7f5308069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530d85f700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5308101a20 0x7f530819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 -- 172.21.15.186:0/519072736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5308101a20 msgr2=0x7f530819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5308101a20 0x7f530819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 -- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53080fff00 msgr2=0x7f5308069480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53080fff00 0x7f5308069480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 -- 172.21.15.186:0/519072736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5300009d00 con 0x7f5308101060 2024-05-08T21:52:55.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530d85f700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5308101a20 0x7f530819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.748+0000 7f530c85d700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 0x7f530819bbe0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f530400b530 tx=0x7f530400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.749+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304011830 con 0x7f5308101060 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.749+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5304011e50 con 0x7f5308101060 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.749+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530819ef80 con 0x7f5308101060 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.749+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530400f2e0 con 0x7f5308101060 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.749+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530819f470 con 0x7f5308101060 2024-05-08T21:52:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.750+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5308107300 con 0x7f5308101060 2024-05-08T21:52:55.754 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.751+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5304010380 con 0x7f5308101060 2024-05-08T21:52:55.754 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.751+0000 7f52fe7fc700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52f40a1120 0x7f52f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:55.754 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.751+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f53040cc4e0 con 0x7f5308101060 2024-05-08T21:52:55.754 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.752+0000 7f530d05e700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52f40a1120 0x7f52f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:55.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.752+0000 7f530d05e700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52f40a1120 0x7f52f40a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009a10 tx=0x7f52f8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:55.756 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.755+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5304096110 con 0x7f5308101060 2024-05-08T21:52:55.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:55 smithi186 ceph-mon[46619]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:55.958 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.957+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f530802cfb0 con 0x7f52f40a1120 2024-05-08T21:52:55.966 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.965+0000 7f52fe7fc700 1 -- 172.21.15.186:0/519072736 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f530802cfb0 con 0x7f52f40a1120 2024-05-08T21:52:55.966 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-05-08T21:52:55.968 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.967+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52f40a1120 msgr2=0x7f52f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:55.968 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.967+0000 7f530f2c2700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52f40a1120 0x7f52f40a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009a10 tx=0x7f52f8009190 comp rx=0 tx=0).stop 2024-05-08T21:52:55.968 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.967+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 msgr2=0x7f530819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:55.968 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.967+0000 7f530f2c2700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 0x7f530819bbe0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f530400b530 tx=0x7f530400b840 comp rx=0 tx=0).stop 2024-05-08T21:52:55.968 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 shutdown_connections 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52f40a1120 0x7f52f40a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5308101a20 0x7f530819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53080fff00 0x7f5308069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 --2- 172.21.15.186:0/519072736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5308101060 0x7f530819bbe0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 >> 172.21.15.186:0/519072736 conn(0x7f53080fb4a0 msgr2=0x7f5308104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 shutdown_connections 2024-05-08T21:52:55.969 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-08T21:52:55.968+0000 7f530f2c2700 1 -- 172.21.15.186:0/519072736 wait complete. 2024-05-08T21:52:56.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:55 smithi160 ceph-mon[46962]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:56.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:55 smithi064 ceph-mon[42500]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:56.329 DEBUG:teuthology.orchestra.run.smithi064:mds.a> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T21:52:56.331 DEBUG:teuthology.orchestra.run.smithi064:mds.d> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T21:52:56.362 INFO:journalctl@ceph.mds.a.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:52:56.363 DEBUG:teuthology.orchestra.run.smithi064:mds.g> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T21:52:56.390 INFO:journalctl@ceph.mds.d.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:52:56.391 DEBUG:teuthology.orchestra.run.smithi064:mds.j> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T21:52:56.419 INFO:journalctl@ceph.mds.g.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:52:56.420 DEBUG:teuthology.orchestra.run.smithi160:mds.b> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T21:52:56.422 DEBUG:teuthology.orchestra.run.smithi160:mds.e> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T21:52:56.447 INFO:journalctl@ceph.mds.j.smithi064.stdout:-- Logs begin at Wed 2024-05-08 21:30:53 UTC. -- 2024-05-08T21:52:56.459 INFO:journalctl@ceph.mds.b.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:52:56.460 DEBUG:teuthology.orchestra.run.smithi160:mds.h> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T21:52:56.488 INFO:journalctl@ceph.mds.e.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:52:56.489 DEBUG:teuthology.orchestra.run.smithi160:mds.k> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T21:52:56.516 INFO:journalctl@ceph.mds.h.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:52:56.517 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T21:52:56.520 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T21:52:56.544 INFO:journalctl@ceph.mds.k.smithi160.stdout:-- Logs begin at Wed 2024-05-08 21:31:05 UTC. -- 2024-05-08T21:52:56.551 INFO:journalctl@ceph.mds.c.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:52:56.553 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T21:52:56.581 INFO:journalctl@ceph.mds.f.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:52:56.582 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T21:52:56.608 INFO:journalctl@ceph.mds.i.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:52:56.609 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T21:52:56.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:52:56.637 INFO:journalctl@ceph.mds.l.smithi186.stdout:-- Logs begin at Wed 2024-05-08 21:31:04 UTC. -- 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- 172.21.15.64:0/1355104565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890106ba0 msgr2=0x7ff890108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- 172.21.15.64:0/1355104565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890106ba0 0x7ff890108fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b0a0 tx=0x7ff88800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- 172.21.15.64:0/1355104565 shutdown_connections 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- 172.21.15.64:0/1355104565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890106ba0 0x7ff890108fb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- 172.21.15.64:0/1355104565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff890101940 0x7ff890103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- 172.21.15.64:0/1355104565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff890104270 0x7ff890106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- 172.21.15.64:0/1355104565 >> 172.21.15.64:0/1355104565 conn(0x7ff8900fb2f0 msgr2=0x7ff8900fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- 172.21.15.64:0/1355104565 shutdown_connections 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- 172.21.15.64:0/1355104565 wait complete. 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 Processor -- start 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- start start 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff890101940 0x7ff890101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 0x7ff8900ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff890106ba0 0x7ff8900ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901005d0 con 0x7ff890104270 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.757+0000 7ff897ad3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890100740 con 0x7ff890101940 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff897ad3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901a0f00 con 0x7ff890106ba0 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 0x7ff8900ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 0x7ff8900ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60588/0 (socket says 172.21.15.64:60588) 2024-05-08T21:52:56.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 -- 172.21.15.64:0/2076406373 learned_addr learned my addr 172.21.15.64:0/2076406373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89586f700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff890101940 0x7ff890101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff896070700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff890106ba0 0x7ff8900ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 -- 172.21.15.64:0/2076406373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff890106ba0 msgr2=0x7ff8900ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff890106ba0 0x7ff8900ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 -- 172.21.15.64:0/2076406373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff890101940 msgr2=0x7ff890101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff890101940 0x7ff890101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 -- 172.21.15.64:0/2076406373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff888009d00 con 0x7ff890104270 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff89506e700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 0x7ff8900ff8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e9a0 tx=0x7ff88000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880009790 con 0x7ff890104270 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88000cb70 con 0x7ff890104270 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8901a1190 con 0x7ff890104270 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880010ea0 con 0x7ff890104270 2024-05-08T21:52:56.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.758+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8901a16a0 con 0x7ff890104270 2024-05-08T21:52:56.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.759+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8900fce70 con 0x7ff890104270 2024-05-08T21:52:56.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.760+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff88000cce0 con 0x7ff890104270 2024-05-08T21:52:56.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.761+0000 7ff886ffd700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff87c0a1120 0x7ff87c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:56.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.761+0000 7ff89586f700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff87c0a1120 0x7ff87c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:56.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.761+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff880014070 con 0x7ff890104270 2024-05-08T21:52:56.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.762+0000 7ff89586f700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff87c0a1120 0x7ff87c0a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009c60 tx=0x7ff88c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:56.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.765+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff880096760 con 0x7ff890104270 2024-05-08T21:52:56.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.975+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff890067c10 con 0x7ff890104270 2024-05-08T21:52:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.976+0000 7ff886ffd700 1 -- 172.21.15.64:0/2076406373 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18422 (secure 0 0 0) 0x7ff880099d80 con 0x7ff890104270 2024-05-08T21:52:56.977 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:52:56.977 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:52:49.704785+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:52:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.978+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff87c0a1120 msgr2=0x7ff87c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.978+0000 7ff897ad3700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff87c0a1120 0x7ff87c0a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009c60 tx=0x7ff88c009260 comp rx=0 tx=0).stop 2024-05-08T21:52:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.978+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 msgr2=0x7ff8900ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.978+0000 7ff897ad3700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 0x7ff8900ff8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e9a0 tx=0x7ff88000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 shutdown_connections 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff87c0a1120 0x7ff87c0a35e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff890101940 0x7ff890101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff890106ba0 0x7ff8900ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 --2- 172.21.15.64:0/2076406373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff890104270 0x7ff8900ff8f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 >> 172.21.15.64:0/2076406373 conn(0x7ff8900fb2f0 msgr2=0x7ff890102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 shutdown_connections 2024-05-08T21:52:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:56.979+0000 7ff897ad3700 1 -- 172.21.15.64:0/2076406373 wait complete. 2024-05-08T21:52:56.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:52:56.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:52:57.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 -- 172.21.15.64:0/3572688660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8100fd0 msgr2=0x7f9ba8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3572688660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8100fd0 0x7f9ba8101450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009990 tx=0x7f9b98009ca0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 -- 172.21.15.64:0/3572688660 shutdown_connections 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3572688660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8101990 0x7f9ba81082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3572688660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba80ffe80 0x7f9ba81002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3572688660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8100fd0 0x7f9ba8101450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 -- 172.21.15.64:0/3572688660 >> 172.21.15.64:0/3572688660 conn(0x7f9ba80fb3f0 msgr2=0x7f9ba80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 -- 172.21.15.64:0/3572688660 shutdown_connections 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.121+0000 7f9bb003d700 1 -- 172.21.15.64:0/3572688660 wait complete. 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 Processor -- start 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 -- start start 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba80ffe80 0x7f9ba8069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba8100fd0 0x7f9ba819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 0x7f9ba819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba819e950 con 0x7f9ba8101990 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba819eac0 con 0x7f9ba80ffe80 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bb003d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba819ec80 con 0x7f9ba8100fd0 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bae5da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 0x7f9ba819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bae5da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 0x7f9ba819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60616/0 (socket says 172.21.15.64:60616) 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bae5da700 1 -- 172.21.15.64:0/3763689011 learned_addr learned my addr 172.21.15.64:0/3763689011 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9baddd9700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba80ffe80 0x7f9ba8069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.122+0000 7f9bad5d8700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba8100fd0 0x7f9ba819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bae5da700 1 -- 172.21.15.64:0/3763689011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba8100fd0 msgr2=0x7f9ba819bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bae5da700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba8100fd0 0x7f9ba819bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bae5da700 1 -- 172.21.15.64:0/3763689011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba80ffe80 msgr2=0x7f9ba8069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bae5da700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba80ffe80 0x7f9ba8069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bae5da700 1 -- 172.21.15.64:0/3763689011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980095f0 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bae5da700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 0x7f9ba819e1b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0007d70 tx=0x7f9ba000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000ad40 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ba00128e0 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba819ef60 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0011780 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.123+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba819f3e0 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.124+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba8107220 con 0x7f9ba8101990 2024-05-08T21:52:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.125+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9ba0012a50 con 0x7f9ba8101990 2024-05-08T21:52:57.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.125+0000 7f9b9effd700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b940a1120 0x7f9b940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.126+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ba00cc760 con 0x7f9ba8101990 2024-05-08T21:52:57.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.126+0000 7f9baddd9700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b940a1120 0x7f9b940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.126+0000 7f9baddd9700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b940a1120 0x7f9b940a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009bf0 tx=0x7f9ba4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:57.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.130+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ba0096390 con 0x7f9ba8101990 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:52:57.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:56 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:57.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.338+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ba810b680 con 0x7f9ba8101990 2024-05-08T21:52:57.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.339+0000 7f9b9effd700 1 -- 172.21.15.64:0/3763689011 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ba00999b0 con 0x7f9ba8101990 2024-05-08T21:52:57.339 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:52:57.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b940a1120 msgr2=0x7f9b940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b940a1120 0x7f9b940a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009bf0 tx=0x7f9ba4009260 comp rx=0 tx=0).stop 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 msgr2=0x7f9ba819e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 0x7f9ba819e1b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0007d70 tx=0x7f9ba000a340 comp rx=0 tx=0).stop 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 shutdown_connections 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9b940a1120 0x7f9b940a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba80ffe80 0x7f9ba8069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ba8101990 0x7f9ba819e1b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 --2- 172.21.15.64:0/3763689011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba8100fd0 0x7f9ba819bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 >> 172.21.15.64:0/3763689011 conn(0x7f9ba80fb3f0 msgr2=0x7f9ba8104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 shutdown_connections 2024-05-08T21:52:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.341+0000 7f9bb003d700 1 -- 172.21.15.64:0/3763689011 wait complete. 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:52:57.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:56 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:57.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:52:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:52:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-08T21:52:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:57.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:52:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:52:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:52:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:52:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:56 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 -- 172.21.15.64:0/3164171397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c102980 msgr2=0x7ffa0c1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/3164171397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c102980 0x7ffa0c1092a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b0a0 tx=0x7ffa0400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 -- 172.21.15.64:0/3164171397 shutdown_connections 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/3164171397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c102980 0x7ffa0c1092a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/3164171397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa0c100e70 0x7ffa0c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/3164171397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 0x7ffa0c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 -- 172.21.15.64:0/3164171397 >> 172.21.15.64:0/3164171397 conn(0x7ffa0c0fc400 msgr2=0x7ffa0c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 -- 172.21.15.64:0/3164171397 shutdown_connections 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 -- 172.21.15.64:0/3164171397 wait complete. 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.480+0000 7ffa11a3d700 1 Processor -- start 2024-05-08T21:52:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 -- start start 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 0x7ffa0c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 0x7ffa0c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa0c102980 0x7ffa0c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c074070 con 0x7ffa0c100e70 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0741e0 con 0x7ffa0c101fc0 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa11a3d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a0f10 con 0x7ffa0c102980 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 0x7ffa0c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 0x7ffa0c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60632/0 (socket says 172.21.15.64:60632) 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 -- 172.21.15.64:0/2908638133 learned_addr learned my addr 172.21.15.64:0/2908638133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0b7fe700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa0c102980 0x7ffa0c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0a7fc700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 0x7ffa0c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 -- 172.21.15.64:0/2908638133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa0c102980 msgr2=0x7ffa0c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa0c102980 0x7ffa0c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 -- 172.21.15.64:0/2908638133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 msgr2=0x7ffa0c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 0x7ffa0c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 -- 172.21.15.64:0/2908638133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009d00 con 0x7ffa0c100e70 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0a7fc700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 0x7ffa0c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa0affd700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 0x7ffa0c074d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9a0 tx=0x7ff9fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.481+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc009790 con 0x7ffa0c100e70 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.482+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc00cb70 con 0x7ffa0c100e70 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.482+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c1a11a0 con 0x7ffa0c100e70 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.482+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc010ea0 con 0x7ffa0c100e70 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.482+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c1a16b0 con 0x7ffa0c100e70 2024-05-08T21:52:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.482+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c108210 con 0x7ffa0c100e70 2024-05-08T21:52:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.484+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff9fc00cce0 con 0x7ffa0c100e70 2024-05-08T21:52:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.484+0000 7ffa10a3b700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff9f40a1120 0x7ff9f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.484+0000 7ffa0a7fc700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff9f40a1120 0x7ff9f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.485+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9fc014070 con 0x7ffa0c100e70 2024-05-08T21:52:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.485+0000 7ffa0a7fc700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff9f40a1120 0x7ff9f40a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000bc10 tx=0x7ffa0000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.488+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9fc096760 con 0x7ffa0c100e70 2024-05-08T21:52:57.667 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 21:52:57 smithi186 systemd[1]: Starting Ceph mds.l for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:52:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.743+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa0c068060 con 0x7ffa0c100e70 2024-05-08T21:52:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.743+0000 7ffa10a3b700 1 -- 172.21.15.64:0/2908638133 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7ff9fc099d80 con 0x7ffa0c100e70 2024-05-08T21:52:57.744 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:52:57.744 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T21:52:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff9f40a1120 msgr2=0x7ff9f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff9f40a1120 0x7ff9f40a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000bc10 tx=0x7ffa0000b5f0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 msgr2=0x7ffa0c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 0x7ffa0c074d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9a0 tx=0x7ff9fc00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 shutdown_connections 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff9f40a1120 0x7ff9f40a35e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa0c102980 0x7ffa0c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa0c100e70 0x7ffa0c074d00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 --2- 172.21.15.64:0/2908638133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa0c101fc0 0x7ffa0c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 >> 172.21.15.64:0/2908638133 conn(0x7ffa0c0fc400 msgr2=0x7ffa0c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 shutdown_connections 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:57.746+0000 7ffa11a3d700 1 -- 172.21.15.64:0/2908638133 wait complete. 2024-05-08T21:52:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 1 2024-05-08T21:52:58.141 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 21:52:58 smithi186 ceph-mds[82662]: -- 172.21.15.186:0/673756962 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a34d8761a0 con 0x55a34d84d000 2024-05-08T21:52:58.141 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 21:52:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l[82639]: starting mds.l at 2024-05-08T21:52:58.141 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 21:52:58 smithi186 bash[82624]: b8393f83253fdad7b12b2f2404d481b3a3115dc7df2ad76eca40dd772b28e1ce 2024-05-08T21:52:58.141 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 21:52:58 smithi186 systemd[1]: Started Ceph mds.l for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:52:58.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:57 smithi186 ceph-mon[46619]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:58.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:57 smithi186 ceph-mon[46619]: Deploying daemon mds.l on smithi186 2024-05-08T21:52:58.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:57 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2076406373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:52:58.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:57 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3763689011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:52:58.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:57 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2908638133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:52:58.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:57 smithi064 ceph-mon[42500]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:57 smithi064 ceph-mon[42500]: Deploying daemon mds.l on smithi186 2024-05-08T21:52:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:57 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2076406373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:52:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:57 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3763689011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:52:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:57 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2908638133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:52:58.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:57 smithi160 ceph-mon[46962]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:58.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:57 smithi160 ceph-mon[46962]: Deploying daemon mds.l on smithi186 2024-05-08T21:52:58.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:57 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2076406373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:52:58.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:57 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3763689011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:52:58.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:57 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2908638133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:52:59.200 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] up:boot 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: fsmap 1 up:standby 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:59.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mon[46962]: Deploying daemon mds.k on smithi160 2024-05-08T21:52:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] up:boot 2024-05-08T21:52:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: fsmap 1 up:standby 2024-05-08T21:52:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:52:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:59.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:52:59 smithi186 ceph-mon[46619]: Deploying daemon mds.k on smithi160 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] up:boot 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: fsmap 1 up:standby 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:52:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:52:59.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:52:59 smithi064 ceph-mon[42500]: Deploying daemon mds.k on smithi160 2024-05-08T21:52:59.544 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 21:52:59 smithi160 systemd[1]: Starting Ceph mds.k for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:52:59.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 -- 172.21.15.64:0/1335166296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b780ffd90 msgr2=0x7f5b781001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/1335166296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b780ffd90 0x7f5b781001b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b68009cd0 comp rx=0 tx=0).stop 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 -- 172.21.15.64:0/1335166296 shutdown_connections 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/1335166296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b78101920 0x7f5b781081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/1335166296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b780ffd90 0x7f5b781001b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/1335166296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 0x7f5b781013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 -- 172.21.15.64:0/1335166296 >> 172.21.15.64:0/1335166296 conn(0x7f5b780fb300 msgr2=0x7f5b780fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 -- 172.21.15.64:0/1335166296 shutdown_connections 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.901+0000 7f5b7cffa700 1 -- 172.21.15.64:0/1335166296 wait complete. 2024-05-08T21:52:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 Processor -- start 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 -- start start 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b780ffd90 0x7f5b78069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 0x7f5b7819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 0x7f5b7819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7819e8a0 con 0x7f5b78101920 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7819ea10 con 0x7f5b78100f80 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7cffa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7819ebd0 con 0x7f5b780ffd90 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 0x7f5b7819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 0x7f5b7819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60648/0 (socket says 172.21.15.64:60648) 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 -- 172.21.15.64:0/3639028283 learned_addr learned my addr 172.21.15.64:0/3639028283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b7659c700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b780ffd90 0x7f5b78069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b75d9b700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 0x7f5b7819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 -- 172.21.15.64:0/3639028283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b780ffd90 msgr2=0x7f5b78069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b780ffd90 0x7f5b78069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 -- 172.21.15.64:0/3639028283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 msgr2=0x7f5b7819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 0x7f5b7819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b76d9d700 1 -- 172.21.15.64:0/3639028283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680095f0 con 0x7f5b78101920 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.902+0000 7f5b75d9b700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 0x7f5b7819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.903+0000 7f5b76d9d700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 0x7f5b7819e100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000ef30 tx=0x7f5b7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.903+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70018070 con 0x7f5b78101920 2024-05-08T21:52:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.903+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7819ee50 con 0x7f5b78101920 2024-05-08T21:52:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.903+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7000f040 con 0x7f5b78101920 2024-05-08T21:52:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.903+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000cdb0 con 0x7f5b78101920 2024-05-08T21:52:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.903+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7819f280 con 0x7f5b78101920 2024-05-08T21:52:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.904+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b78107260 con 0x7f5b78101920 2024-05-08T21:52:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.905+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5b70003a00 con 0x7f5b78101920 2024-05-08T21:52:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.906+0000 7f5b677fe700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5b600a1120 0x7f5b600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:52:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.906+0000 7f5b7659c700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5b600a1120 0x7f5b600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:52:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.906+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b700cc980 con 0x7f5b78101920 2024-05-08T21:52:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.906+0000 7f5b7659c700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5b600a1120 0x7f5b600a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b68003730 tx=0x7f5b68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:52:59.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:52:59.910+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5b70096640 con 0x7f5b78101920 2024-05-08T21:53:00.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.172+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b78067380 con 0x7f5b78101920 2024-05-08T21:53:00.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.172+0000 7f5b677fe700 1 -- 172.21.15.64:0/3639028283 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f5b70099c60 con 0x7f5b78101920 2024-05-08T21:53:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:00.173 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T21:53:00.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5b600a1120 msgr2=0x7f5b600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:00.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5b600a1120 0x7f5b600a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b68003730 tx=0x7f5b68000bc0 comp rx=0 tx=0).stop 2024-05-08T21:53:00.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 msgr2=0x7f5b7819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 0x7f5b7819e100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000ef30 tx=0x7f5b7000c3a0 comp rx=0 tx=0).stop 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 shutdown_connections 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5b600a1120 0x7f5b600a35e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b78101920 0x7f5b7819e100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b780ffd90 0x7f5b78069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 --2- 172.21.15.64:0/3639028283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78100f80 0x7f5b7819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 >> 172.21.15.64:0/3639028283 conn(0x7f5b780fb300 msgr2=0x7f5b781043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 shutdown_connections 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:00.175+0000 7f5b7cffa700 1 -- 172.21.15.64:0/3639028283 wait complete. 2024-05-08T21:53:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-05-08T21:53:00.294 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 21:52:59 smithi160 ceph-mds[81955]: -- 172.21.15.160:0/3889785669 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ed072c1a0 con 0x562ed0703000 2024-05-08T21:53:00.295 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 21:52:59 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k[81932]: starting mds.k at 2024-05-08T21:53:00.295 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 21:52:59 smithi160 bash[81918]: 21468ab5befbaaa9ac971355f8d60a646b3b97334fc7516450d4fc32cf7963b1 2024-05-08T21:53:00.295 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 21:52:59 smithi160 systemd[1]: Started Ceph mds.k for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: Deploying daemon mds.j on smithi064 2024-05-08T21:53:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] up:boot 2024-05-08T21:53:01.086 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: fsmap 2 up:standby 2024-05-08T21:53:01.086 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:53:01.086 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:00 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3639028283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: Deploying daemon mds.j on smithi064 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] up:boot 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: fsmap 2 up:standby 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:53:01.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:00 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3639028283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:01.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: Deploying daemon mds.j on smithi064 2024-05-08T21:53:01.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] up:boot 2024-05-08T21:53:01.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: fsmap 2 up:standby 2024-05-08T21:53:01.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:53:01.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:00 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3639028283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:01.431 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 21:53:01 smithi064 systemd[1]: Starting Ceph mds.j for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:01.892 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 21:53:01 smithi064 bash[93563]: 3e49a748744474c83309fb513bc77b4455603e08ad6b95af1bb61ee924434996 2024-05-08T21:53:01.892 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 21:53:01 smithi064 systemd[1]: Started Ceph mds.j for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:01.892 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mds[93601]: -- 172.21.15.64:0/1798792321 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ecafd21a0 con 0x561ecafa8800 2024-05-08T21:53:01.892 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 21:53:01 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-j[93578]: starting mds.j at 2024-05-08T21:53:02.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:02.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:02.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:02.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:01 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:02.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:02.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:01 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 -- 172.21.15.64:0/571442583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c81018f0 msgr2=0x7f27c81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 --2- 172.21.15.64:0/571442583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c81018f0 0x7f27c81081a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b0d0 tx=0x7f27c000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 -- 172.21.15.64:0/571442583 shutdown_connections 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 --2- 172.21.15.64:0/571442583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27c80ffd60 0x7f27c8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 --2- 172.21.15.64:0/571442583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c81018f0 0x7f27c81081a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 --2- 172.21.15.64:0/571442583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100f50 0x7f27c81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 -- 172.21.15.64:0/571442583 >> 172.21.15.64:0/571442583 conn(0x7f27c80fb2f0 msgr2=0x7f27c80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 -- 172.21.15.64:0/571442583 shutdown_connections 2024-05-08T21:53:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.324+0000 7f27cfb52700 1 -- 172.21.15.64:0/571442583 wait complete. 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 Processor -- start 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 -- start start 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 0x7f27c8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100f50 0x7f27c81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27c81018f0 0x7f27c819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819d8e0 con 0x7f27c80ffd60 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819da20 con 0x7f27c81018f0 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cfb52700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819dbb0 con 0x7f27c8100f50 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cd8ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 0x7f27c8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cd8ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 0x7f27c8195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60690/0 (socket says 172.21.15.64:60690) 2024-05-08T21:53:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.325+0000 7f27cd8ee700 1 -- 172.21.15.64:0/474181211 learned_addr learned my addr 172.21.15.64:0/474181211 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd0ed700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100f50 0x7f27c81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27ce0ef700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27c81018f0 0x7f27c819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd8ee700 1 -- 172.21.15.64:0/474181211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100f50 msgr2=0x7f27c81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd8ee700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100f50 0x7f27c81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd8ee700 1 -- 172.21.15.64:0/474181211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27c81018f0 msgr2=0x7f27c819d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd8ee700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27c81018f0 0x7f27c819d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd8ee700 1 -- 172.21.15.64:0/474181211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0009d00 con 0x7f27c80ffd60 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cd8ee700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 0x7f27c8195db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c970 tx=0x7f27b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004220 con 0x7f27c80ffd60 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b8004380 con 0x7f27c80ffd60 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c819de90 con 0x7f27c80ffd60 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8003b50 con 0x7f27c80ffd60 2024-05-08T21:53:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.326+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c819e2b0 con 0x7f27c80ffd60 2024-05-08T21:53:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.327+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8067c80 con 0x7f27c80ffd60 2024-05-08T21:53:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.329+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f27b8003cd0 con 0x7f27c80ffd60 2024-05-08T21:53:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.330+0000 7f27beffd700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27b40a1120 0x7f27b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.330+0000 7f27cd0ed700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27b40a1120 0x7f27b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:02.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.330+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27b80cc0e0 con 0x7f27c80ffd60 2024-05-08T21:53:02.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.330+0000 7f27cd0ed700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27b40a1120 0x7f27b40a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005950 tx=0x7f27c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.333+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f27b8095d10 con 0x7f27c80ffd60 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:01 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.598+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27c8107110 con 0x7f27c80ffd60 2024-05-08T21:53:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.598+0000 7f27beffd700 1 -- 172.21.15.64:0/474181211 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f27b8099330 con 0x7f27c80ffd60 2024-05-08T21:53:02.599 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:02.599 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T21:53:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.600+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27b40a1120 msgr2=0x7f27b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.600+0000 7f27cfb52700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27b40a1120 0x7f27b40a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005950 tx=0x7f27c400b410 comp rx=0 tx=0).stop 2024-05-08T21:53:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 msgr2=0x7f27c8195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 0x7f27c8195db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c970 tx=0x7f27b800cc80 comp rx=0 tx=0).stop 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 shutdown_connections 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27b40a1120 0x7f27b40a35e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27c81018f0 0x7f27c819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c80ffd60 0x7f27c8195db0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 --2- 172.21.15.64:0/474181211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100f50 0x7f27c81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 >> 172.21.15.64:0/474181211 conn(0x7f27c80fb2f0 msgr2=0x7f27c8104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 shutdown_connections 2024-05-08T21:53:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:02.601+0000 7f27cfb52700 1 -- 172.21.15.64:0/474181211 wait complete. 2024-05-08T21:53:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 4 2024-05-08T21:53:03.213 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 21:53:02 smithi064 systemd[1]: Starting Ceph mds.d for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:03.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:02 smithi064 ceph-mon[42500]: Deploying daemon mds.d on smithi064 2024-05-08T21:53:03.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:02 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.64:6834/1013310414,v1:172.21.15.64:6835/1013310414] up:boot 2024-05-08T21:53:03.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:02 smithi064 ceph-mon[42500]: fsmap 3 up:standby 2024-05-08T21:53:03.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:02 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:53:03.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/474181211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:03.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:02 smithi160 ceph-mon[46962]: Deploying daemon mds.d on smithi064 2024-05-08T21:53:03.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:02 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.64:6834/1013310414,v1:172.21.15.64:6835/1013310414] up:boot 2024-05-08T21:53:03.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:02 smithi160 ceph-mon[46962]: fsmap 3 up:standby 2024-05-08T21:53:03.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:02 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:53:03.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/474181211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:02 smithi186 ceph-mon[46619]: Deploying daemon mds.d on smithi064 2024-05-08T21:53:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:02 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.64:6834/1013310414,v1:172.21.15.64:6835/1013310414] up:boot 2024-05-08T21:53:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:02 smithi186 ceph-mon[46619]: fsmap 3 up:standby 2024-05-08T21:53:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:02 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:53:03.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/474181211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:03.713 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mds[93987]: -- 172.21.15.64:0/61284800 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0272a01a0 con 0x55b027277000 2024-05-08T21:53:03.713 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 21:53:03 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-d[93963]: starting mds.d at 2024-05-08T21:53:03.713 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 21:53:03 smithi064 bash[93948]: e9211df1fa6122c4c6bfae7ea976c99d5292541503b9526ce04ffe67542c2764 2024-05-08T21:53:03.713 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 21:53:03 smithi064 systemd[1]: Started Ceph mds.d for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:04.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:04.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:04.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:04.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:03 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:03 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:04.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:04.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:04.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:03 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:04.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 -- 172.21.15.64:0/2750510166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 msgr2=0x7fa17c1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 --2- 172.21.15.64:0/2750510166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c1001c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0099c0 tx=0x7fa16c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 -- 172.21.15.64:0/2750510166 shutdown_connections 2024-05-08T21:53:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 --2- 172.21.15.64:0/2750510166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c101930 0x7fa17c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 --2- 172.21.15.64:0/2750510166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c1001c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 --2- 172.21.15.64:0/2750510166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 0x7fa17c1013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 -- 172.21.15.64:0/2750510166 >> 172.21.15.64:0/2750510166 conn(0x7fa17c0fb330 msgr2=0x7fa17c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 -- 172.21.15.64:0/2750510166 shutdown_connections 2024-05-08T21:53:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.760+0000 7fa1815f1700 1 -- 172.21.15.64:0/2750510166 wait complete. 2024-05-08T21:53:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.761+0000 7fa1815f1700 1 Processor -- start 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.761+0000 7fa1815f1700 1 -- start start 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.761+0000 7fa1815f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.761+0000 7fa1815f1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 0x7fa17c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.761+0000 7fa1815f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c101930 0x7fa17c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.761+0000 7fa1815f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c19d830 con 0x7fa17c0ffda0 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa1815f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c19d9a0 con 0x7fa17c101930 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa1815f1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c19db60 con 0x7fa17c100f90 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58948/0 (socket says 172.21.15.64:58948) 2024-05-08T21:53:04.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 -- 172.21.15.64:0/292705778 learned_addr learned my addr 172.21.15.64:0/292705778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17b7fe700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c101930 0x7fa17c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17a7fc700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 0x7fa17c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 -- 172.21.15.64:0/292705778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 msgr2=0x7fa17c196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 0x7fa17c196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 -- 172.21.15.64:0/292705778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c101930 msgr2=0x7fa17c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c101930 0x7fa17c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 -- 172.21.15.64:0/292705778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0095f0 con 0x7fa17c0ffda0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17a7fc700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 0x7fa17c196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa17affd700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c195d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa16c003820 tx=0x7fa16c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01d070 con 0x7fa17c0ffda0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c19dde0 con 0x7fa17c0ffda0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16c0054f0 con 0x7fa17c0ffda0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c004a00 con 0x7fa17c0ffda0 2024-05-08T21:53:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.762+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c19e1d0 con 0x7fa17c0ffda0 2024-05-08T21:53:04.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.763+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c1071c0 con 0x7fa17c0ffda0 2024-05-08T21:53:04.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.765+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa16c003b50 con 0x7fa17c0ffda0 2024-05-08T21:53:04.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.766+0000 7fa173fff700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680a1120 0x7fa1680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:04.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.766+0000 7fa17a7fc700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680a1120 0x7fa1680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:04.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.766+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa16c0cd660 con 0x7fa17c0ffda0 2024-05-08T21:53:04.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.766+0000 7fa17a7fc700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680a1120 0x7fa1680a35e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa164007900 tx=0x7fa164008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:04.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:04.769+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa16c097320 con 0x7fa17c0ffda0 2024-05-08T21:53:05.025 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:04 smithi160 ceph-mon[46962]: Deploying daemon mds.e on smithi160 2024-05-08T21:53:05.025 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:04 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.64:6836/2682658639,v1:172.21.15.64:6837/2682658639] up:boot 2024-05-08T21:53:05.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:04 smithi160 ceph-mon[46962]: fsmap 4 up:standby 2024-05-08T21:53:05.026 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:04 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:53:05.026 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 21:53:04 smithi160 systemd[1]: Starting Ceph mds.e for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:05.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.025+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa17c068060 con 0x7fa17c0ffda0 2024-05-08T21:53:05.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.025+0000 7fa173fff700 1 -- 172.21.15.64:0/292705778 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7fa16c09a940 con 0x7fa17c0ffda0 2024-05-08T21:53:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:05.027 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T21:53:05.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680a1120 msgr2=0x7fa1680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:05.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680a1120 0x7fa1680a35e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa164007900 tx=0x7fa164008040 comp rx=0 tx=0).stop 2024-05-08T21:53:05.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 msgr2=0x7fa17c195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:05.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c195d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa16c003820 tx=0x7fa16c004730 comp rx=0 tx=0).stop 2024-05-08T21:53:05.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 shutdown_connections 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa1680a1120 0x7fa1680a35e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c101930 0x7fa17c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0ffda0 0x7fa17c195d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 --2- 172.21.15.64:0/292705778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa17c100f90 0x7fa17c196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.027+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 >> 172.21.15.64:0/292705778 conn(0x7fa17c0fb330 msgr2=0x7fa17c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.028+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 shutdown_connections 2024-05-08T21:53:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:05.028+0000 7fa1815f1700 1 -- 172.21.15.64:0/292705778 wait complete. 2024-05-08T21:53:05.030 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 5 2024-05-08T21:53:05.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:04 smithi064 ceph-mon[42500]: Deploying daemon mds.e on smithi160 2024-05-08T21:53:05.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:04 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.64:6836/2682658639,v1:172.21.15.64:6837/2682658639] up:boot 2024-05-08T21:53:05.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:04 smithi064 ceph-mon[42500]: fsmap 4 up:standby 2024-05-08T21:53:05.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:04 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:53:05.294 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mds[82306]: -- 172.21.15.160:0/3892927857 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3507121a0 con 0x55c3506e8c00 2024-05-08T21:53:05.295 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 21:53:05 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e[82283]: starting mds.e at 2024-05-08T21:53:05.295 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 21:53:05 smithi160 bash[82268]: 20b940966dffc677578bd022f09c810633f8d8f46a7a0ce8ae187359b7656548 2024-05-08T21:53:05.295 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 21:53:05 smithi160 systemd[1]: Started Ceph mds.e for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:04 smithi186 ceph-mon[46619]: Deploying daemon mds.e on smithi160 2024-05-08T21:53:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:04 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.64:6836/2682658639,v1:172.21.15.64:6837/2682658639] up:boot 2024-05-08T21:53:05.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:04 smithi186 ceph-mon[46619]: fsmap 4 up:standby 2024-05-08T21:53:05.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:04 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/292705778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:05 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/292705778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:05 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/292705778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:05 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:06.410 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 21:53:06 smithi186 systemd[1]: Starting Ceph mds.f for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:07.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:07.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: Deploying daemon mds.f on smithi186 2024-05-08T21:53:07.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6834/770617410,v1:172.21.15.160:6835/770617410] up:boot 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: fsmap 5 up:standby 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 21:53:06 smithi186 ceph-mds[83007]: -- 172.21.15.186:0/4001774150 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560dee0f21a0 con 0x560dee0c8c00 2024-05-08T21:53:07.142 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 21:53:06 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f[82984]: starting mds.f at 2024-05-08T21:53:07.143 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 21:53:06 smithi186 bash[82969]: 3b54f91c877e1f9dc034bc6618cafdf8453427b7f187d705645f760b9bca0df8 2024-05-08T21:53:07.143 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 21:53:06 smithi186 systemd[1]: Started Ceph mds.f for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:07.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.180+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1875458540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc106be0 msgr2=0x7f66bc108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:07.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.180+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1875458540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc106be0 0x7f66bc108fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66b400b0a0 tx=0x7f66b400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.180+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1875458540 shutdown_connections 2024-05-08T21:53:07.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.180+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1875458540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc101980 0x7f66bc103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1875458540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc106be0 0x7f66bc108fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1875458540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66bc1042b0 0x7f66bc1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1875458540 >> 172.21.15.64:0/1875458540 conn(0x7f66bc0fb2f0 msgr2=0x7f66bc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:07.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1875458540 shutdown_connections 2024-05-08T21:53:07.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1875458540 wait complete. 2024-05-08T21:53:07.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 Processor -- start 2024-05-08T21:53:07.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 -- start start 2024-05-08T21:53:07.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66bc101980 0x7f66bc196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:07.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.181+0000 7f66c1ee1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 0x7f66bc197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:07.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66c1ee1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc106be0 0x7f66bc19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66c1ee1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc19ea80 con 0x7f66bc1042b0 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66c1ee1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc19ebc0 con 0x7f66bc106be0 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66c1ee1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc19ed50 con 0x7f66bc101980 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 0x7f66bc197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 0x7f66bc197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58962/0 (socket says 172.21.15.64:58962) 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66bb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66bc101980 0x7f66bc196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 -- 172.21.15.64:0/1721534779 learned_addr learned my addr 172.21.15.64:0/1721534779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66bbfff700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc106be0 0x7f66bc19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 -- 172.21.15.64:0/1721534779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66bc101980 msgr2=0x7f66bc196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66bc101980 0x7f66bc196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 -- 172.21.15.64:0/1721534779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc106be0 msgr2=0x7f66bc19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc106be0 0x7f66bc19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 -- 172.21.15.64:0/1721534779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b4009d00 con 0x7f66bc1042b0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66bbfff700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc106be0 0x7f66bc19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66baffd700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 0x7f66bc197490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00b530 tx=0x7f66ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac011830 con 0x7f66bc1042b0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac011e50 con 0x7f66bc1042b0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc19efd0 con 0x7f66bc1042b0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00f2e0 con 0x7f66bc1042b0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.182+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc19f4b0 con 0x7f66bc1042b0 2024-05-08T21:53:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.183+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc0fce70 con 0x7f66bc1042b0 2024-05-08T21:53:07.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.184+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f66ac010380 con 0x7f66bc1042b0 2024-05-08T21:53:07.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.185+0000 7f66b8ff9700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f66a40a1120 0x7f66a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:07.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.185+0000 7f66bb7fe700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f66a40a1120 0x7f66a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:07.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.185+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66ac0cbf90 con 0x7f66bc1042b0 2024-05-08T21:53:07.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.186+0000 7f66bb7fe700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f66a40a1120 0x7f66a40a35e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f66b0007c60 tx=0x7f66b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:07.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.190+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f66ac0ffd60 con 0x7f66bc1042b0 2024-05-08T21:53:07.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: Deploying daemon mds.f on smithi186 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6834/770617410,v1:172.21.15.160:6835/770617410] up:boot 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: fsmap 5 up:standby 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:07.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:06 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:07.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: Deploying daemon mds.f on smithi186 2024-05-08T21:53:07.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6834/770617410,v1:172.21.15.160:6835/770617410] up:boot 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: fsmap 5 up:standby 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:06 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:07.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.444+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66bc067c10 con 0x7f66bc1042b0 2024-05-08T21:53:07.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.445+0000 7f66b8ff9700 1 -- 172.21.15.64:0/1721534779 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5051 (secure 0 0 0) 0x7f66ac095bc0 con 0x7f66bc1042b0 2024-05-08T21:53:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:07.445 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-05-08T21:53:07.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f66a40a1120 msgr2=0x7f66a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:07.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f66a40a1120 0x7f66a40a35e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f66b0007c60 tx=0x7f66b00073d0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 msgr2=0x7f66bc197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 0x7f66bc197490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00b530 tx=0x7f66ac00b840 comp rx=0 tx=0).stop 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 shutdown_connections 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f66a40a1120 0x7f66a40a35e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc106be0 0x7f66bc19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66bc101980 0x7f66bc196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 --2- 172.21.15.64:0/1721534779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc1042b0 0x7f66bc197490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 >> 172.21.15.64:0/1721534779 conn(0x7f66bc0fb2f0 msgr2=0x7f66bc102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 shutdown_connections 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:07.447+0000 7f66c1ee1700 1 -- 172.21.15.64:0/1721534779 wait complete. 2024-05-08T21:53:07.448 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-05-08T21:53:08.107 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 21:53:08 smithi186 systemd[1]: Starting Ceph mds.c for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:08.108 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:07 smithi186 ceph-mon[46619]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:08.108 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:07 smithi186 ceph-mon[46619]: Deploying daemon mds.c on smithi186 2024-05-08T21:53:08.108 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:07 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6834/4069999836,v1:172.21.15.186:6835/4069999836] up:boot 2024-05-08T21:53:08.108 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:07 smithi186 ceph-mon[46619]: fsmap 6 up:standby 2024-05-08T21:53:08.108 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:07 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:53:08.108 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:07 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1721534779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:08.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:07 smithi064 ceph-mon[42500]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:08.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:07 smithi064 ceph-mon[42500]: Deploying daemon mds.c on smithi186 2024-05-08T21:53:08.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:07 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6834/4069999836,v1:172.21.15.186:6835/4069999836] up:boot 2024-05-08T21:53:08.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:07 smithi064 ceph-mon[42500]: fsmap 6 up:standby 2024-05-08T21:53:08.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:07 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:53:08.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:07 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1721534779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:07 smithi160 ceph-mon[46962]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:07 smithi160 ceph-mon[46962]: Deploying daemon mds.c on smithi186 2024-05-08T21:53:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:07 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6834/4069999836,v1:172.21.15.186:6835/4069999836] up:boot 2024-05-08T21:53:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:07 smithi160 ceph-mon[46962]: fsmap 6 up:standby 2024-05-08T21:53:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:07 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:53:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:07 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1721534779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:08.891 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 21:53:08 smithi186 ceph-mds[83358]: -- 172.21.15.186:0/1529434073 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c324f0e1a0 con 0x55c324ee4800 2024-05-08T21:53:08.891 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 21:53:08 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c[83335]: starting mds.c at 2024-05-08T21:53:08.892 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 21:53:08 smithi186 bash[83320]: 83c94909c57be323fd87ebf224fbdd3590112593b8f0257dd99cf259280e9d26 2024-05-08T21:53:08.892 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 21:53:08 smithi186 systemd[1]: Started Ceph mds.c for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:09.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 -- 172.21.15.64:0/3342234592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc100f80 msgr2=0x7f73fc1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 --2- 172.21.15.64:0/3342234592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc100f80 0x7f73fc1013e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73f00099c0 tx=0x7f73f0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 -- 172.21.15.64:0/3342234592 shutdown_connections 2024-05-08T21:53:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 --2- 172.21.15.64:0/3342234592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc101920 0x7f73fc1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 --2- 172.21.15.64:0/3342234592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc0ffd90 0x7f73fc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 --2- 172.21.15.64:0/3342234592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc100f80 0x7f73fc1013e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 -- 172.21.15.64:0/3342234592 >> 172.21.15.64:0/3342234592 conn(0x7f73fc0fb300 msgr2=0x7f73fc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 -- 172.21.15.64:0/3342234592 shutdown_connections 2024-05-08T21:53:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.602+0000 7f7400b0c700 1 -- 172.21.15.64:0/3342234592 wait complete. 2024-05-08T21:53:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 Processor -- start 2024-05-08T21:53:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 -- start start 2024-05-08T21:53:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 0x7f73fc196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc100f80 0x7f73fc197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc101920 0x7f73fc19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc19e970 con 0x7f73fc0ffd90 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc19eab0 con 0x7f73fc101920 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.603+0000 7f7400b0c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc19ec40 con 0x7f73fc100f80 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 0x7f73fc196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 0x7f73fc196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58984/0 (socket says 172.21.15.64:58984) 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 -- 172.21.15.64:0/1443713657 learned_addr learned my addr 172.21.15.64:0/1443713657 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fad9d700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc101920 0x7f73fc19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73f9d9b700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc100f80 0x7f73fc197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 -- 172.21.15.64:0/1443713657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc100f80 msgr2=0x7f73fc197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc100f80 0x7f73fc197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 -- 172.21.15.64:0/1443713657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc101920 msgr2=0x7f73fc19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc101920 0x7f73fc19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 -- 172.21.15.64:0/1443713657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f00095f0 con 0x7f73fc0ffd90 2024-05-08T21:53:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73fa59c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 0x7f73fc196e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00d980 tx=0x7f73ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec004020 con 0x7f73fc0ffd90 2024-05-08T21:53:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73ec00f690 con 0x7f73fc0ffd90 2024-05-08T21:53:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc19ef20 con 0x7f73fc0ffd90 2024-05-08T21:53:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec003a60 con 0x7f73fc0ffd90 2024-05-08T21:53:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.604+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc19f340 con 0x7f73fc0ffd90 2024-05-08T21:53:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.605+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc067380 con 0x7f73fc0ffd90 2024-05-08T21:53:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.606+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f73ec00fb00 con 0x7f73fc0ffd90 2024-05-08T21:53:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.607+0000 7f73eb7fe700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73e40a1120 0x7f73e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.607+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73ec0cc320 con 0x7f73fc0ffd90 2024-05-08T21:53:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.607+0000 7f73f9d9b700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73e40a1120 0x7f73e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.607+0000 7f73f9d9b700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73e40a1120 0x7f73e40a35e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73f0003730 tx=0x7f73f000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:09.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.610+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f73ec095f50 con 0x7f73fc0ffd90 2024-05-08T21:53:09.633 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: Deploying daemon mds.h on smithi160 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6836/3888486173,v1:172.21.15.186:6837/3888486173] up:boot 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: fsmap 7 up:standby 2024-05-08T21:53:09.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:09 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:53:09.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.868+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73fc10b8a0 con 0x7f73fc0ffd90 2024-05-08T21:53:09.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.869+0000 7f73eb7fe700 1 -- 172.21.15.64:0/1443713657 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5811 (secure 0 0 0) 0x7f73ec099570 con 0x7f73fc0ffd90 2024-05-08T21:53:09.870 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:09.870 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T21:53:09.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73e40a1120 msgr2=0x7f73e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73e40a1120 0x7f73e40a35e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73f0003730 tx=0x7f73f000b540 comp rx=0 tx=0).stop 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 msgr2=0x7f73fc196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 0x7f73fc196e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00d980 tx=0x7f73ec00dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 shutdown_connections 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73e40a1120 0x7f73e40a35e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc101920 0x7f73fc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0ffd90 0x7f73fc196e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 --2- 172.21.15.64:0/1443713657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc100f80 0x7f73fc197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 >> 172.21.15.64:0/1443713657 conn(0x7f73fc0fb300 msgr2=0x7f73fc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 shutdown_connections 2024-05-08T21:53:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:09.871+0000 7f7400b0c700 1 -- 172.21.15.64:0/1443713657 wait complete. 2024-05-08T21:53:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-05-08T21:53:09.882 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: Deploying daemon mds.h on smithi160 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6836/3888486173,v1:172.21.15.186:6837/3888486173] up:boot 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: fsmap 7 up:standby 2024-05-08T21:53:09.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:09 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: Deploying daemon mds.h on smithi160 2024-05-08T21:53:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6836/3888486173,v1:172.21.15.186:6837/3888486173] up:boot 2024-05-08T21:53:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: fsmap 7 up:standby 2024-05-08T21:53:09.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:09 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:53:10.177 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 21:53:09 smithi160 systemd[1]: Starting Ceph mds.h for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:10.794 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mds[82646]: -- 172.21.15.160:0/270173274 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dbfe22e1a0 con 0x55dbfe204800 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 21:53:10 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h[82623]: starting mds.h at 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 21:53:10 smithi160 bash[82608]: b50d12d54e9f0ea294e148af7614b7463e8457d353cd4c21c7831f042735cd72 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 21:53:10 smithi160 systemd[1]: Started Ceph mds.h for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1443713657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:10.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:10.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:10 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:10.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1443713657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:10.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:10.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:10.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:10 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1443713657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:10 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:11.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:11 smithi186 ceph-mon[46619]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:11.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:11 smithi186 ceph-mon[46619]: Deploying daemon mds.i on smithi186 2024-05-08T21:53:11.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:11 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] up:boot 2024-05-08T21:53:11.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:11 smithi186 ceph-mon[46619]: fsmap 8 up:standby 2024-05-08T21:53:11.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:11 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:53:11.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:11 smithi160 ceph-mon[46962]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:11.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:11 smithi160 ceph-mon[46962]: Deploying daemon mds.i on smithi186 2024-05-08T21:53:11.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:11 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] up:boot 2024-05-08T21:53:11.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:11 smithi160 ceph-mon[46962]: fsmap 8 up:standby 2024-05-08T21:53:11.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:11 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:53:11.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:11.915 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:11 smithi064 ceph-mon[42500]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:11.916 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:11 smithi064 ceph-mon[42500]: Deploying daemon mds.i on smithi186 2024-05-08T21:53:11.916 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:11 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] up:boot 2024-05-08T21:53:11.916 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:11 smithi064 ceph-mon[42500]: fsmap 8 up:standby 2024-05-08T21:53:11.916 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:11 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:53:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 -- 172.21.15.64:0/2715461717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918101920 msgr2=0x7f79181081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 --2- 172.21.15.64:0/2715461717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918101920 0x7f79181081d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f791000b0d0 tx=0x7f791000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 -- 172.21.15.64:0/2715461717 shutdown_connections 2024-05-08T21:53:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 --2- 172.21.15.64:0/2715461717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79180ffd90 0x7f79181001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 --2- 172.21.15.64:0/2715461717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918101920 0x7f79181081d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 --2- 172.21.15.64:0/2715461717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7918100f80 0x7f79181013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 -- 172.21.15.64:0/2715461717 >> 172.21.15.64:0/2715461717 conn(0x7f79180fb300 msgr2=0x7f79180fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 -- 172.21.15.64:0/2715461717 shutdown_connections 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.025+0000 7f791ee10700 1 -- 172.21.15.64:0/2715461717 wait complete. 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 Processor -- start 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 -- start start 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79180ffd90 0x7f7918196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 0x7f79181974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101920 0x7f791819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791819ea40 con 0x7f7918100f80 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791819eb80 con 0x7f7918101920 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791ee10700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791819ed10 con 0x7f79180ffd90 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f7917fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 0x7f79181974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f7917fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 0x7f79181974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59008/0 (socket says 172.21.15.64:59008) 2024-05-08T21:53:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f7917fff700 1 -- 172.21.15.64:0/676988151 learned_addr learned my addr 172.21.15.64:0/676988151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791cbac700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79180ffd90 0x7f7918196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.026+0000 7f791d3ad700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101920 0x7f791819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7917fff700 1 -- 172.21.15.64:0/676988151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79180ffd90 msgr2=0x7f7918196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7917fff700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79180ffd90 0x7f7918196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7917fff700 1 -- 172.21.15.64:0/676988151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101920 msgr2=0x7f791819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7917fff700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101920 0x7f791819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7917fff700 1 -- 172.21.15.64:0/676988151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910009d00 con 0x7f7918100f80 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f791d3ad700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101920 0x7f791819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7917fff700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 0x7f79181974a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9a0 tx=0x7f790400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904009790 con 0x7f7918100f80 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f790400cbd0 con 0x7f7918100f80 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791819eff0 con 0x7f7918100f80 2024-05-08T21:53:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904018680 con 0x7f7918100f80 2024-05-08T21:53:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.027+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791819f4c0 con 0x7f7918100f80 2024-05-08T21:53:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.028+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918067380 con 0x7f7918100f80 2024-05-08T21:53:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.030+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f790400cd40 con 0x7f7918100f80 2024-05-08T21:53:12.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.031+0000 7f7915ffb700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f79080a11a0 0x7f79080a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:12.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.031+0000 7f791cbac700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f79080a11a0 0x7f79080a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:12.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.031+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7904014070 con 0x7f7918100f80 2024-05-08T21:53:12.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.031+0000 7f791cbac700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f79080a11a0 0x7f79080a3660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7918100de0 tx=0x7f790c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.034+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7904096550 con 0x7f7918100f80 2024-05-08T21:53:12.075 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 21:53:11 smithi186 systemd[1]: Starting Ceph mds.i for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:12.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.290+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f791810b870 con 0x7f7918100f80 2024-05-08T21:53:12.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.291+0000 7f7915ffb700 1 -- 172.21.15.64:0/676988151 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6571 (secure 0 0 0) 0x7f7904099b70 con 0x7f7918100f80 2024-05-08T21:53:12.291 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:12.292 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f79080a11a0 msgr2=0x7f79080a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f79080a11a0 0x7f79080a3660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7918100de0 tx=0x7f790c009260 comp rx=0 tx=0).stop 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 msgr2=0x7f79181974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 0x7f79181974a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9a0 tx=0x7f790400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 shutdown_connections 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f79080a11a0 0x7f79080a3660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101920 0x7f791819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79180ffd90 0x7f7918196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 --2- 172.21.15.64:0/676988151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7918100f80 0x7f79181974a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 >> 172.21.15.64:0/676988151 conn(0x7f79180fb300 msgr2=0x7f79180fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 shutdown_connections 2024-05-08T21:53:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:12.293+0000 7f791ee10700 1 -- 172.21.15.64:0/676988151 wait complete. 2024-05-08T21:53:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 9 2024-05-08T21:53:12.391 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 21:53:12 smithi186 ceph-mds[83701]: -- 172.21.15.186:0/1548229170 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605608621a0 con 0x560560838c00 2024-05-08T21:53:12.391 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 21:53:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i[83678]: starting mds.i at 2024-05-08T21:53:12.391 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 21:53:12 smithi186 bash[83663]: 573eaf21e78a1315c4ccc69b346e29144c388c1da8b4ab3ed1672f190b0b9834 2024-05-08T21:53:12.391 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 21:53:12 smithi186 systemd[1]: Started Ceph mds.i for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:13.293 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: Deploying daemon mds.g on smithi064 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/676988151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] up:boot 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: fsmap 9 up:standby 2024-05-08T21:53:13.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T21:53:13.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:13.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: Deploying daemon mds.g on smithi064 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/676988151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] up:boot 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: fsmap 9 up:standby 2024-05-08T21:53:13.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:13 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T21:53:13.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: Deploying daemon mds.g on smithi064 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/676988151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] up:boot 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: fsmap 9 up:standby 2024-05-08T21:53:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:13 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T21:53:13.826 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 21:53:13 smithi064 systemd[1]: Starting Ceph mds.g for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:14.213 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 21:53:13 smithi064 ceph-mds[94516]: -- 172.21.15.64:0/1354162708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5585f58541a0 con 0x5585f582a800 2024-05-08T21:53:14.213 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 21:53:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g[94493]: starting mds.g at 2024-05-08T21:53:14.213 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 21:53:13 smithi064 bash[94478]: d0e9ce30abc15951751abd943a4004372e09eab66290c63094c4528431cd59af 2024-05-08T21:53:14.213 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 21:53:13 smithi064 systemd[1]: Started Ceph mds.g for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:14.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 -- 172.21.15.64:0/1188302884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac102980 msgr2=0x7f1dac1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 --2- 172.21.15.64:0/1188302884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac102980 0x7f1dac1092a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b0a0 tx=0x7f1da400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 -- 172.21.15.64:0/1188302884 shutdown_connections 2024-05-08T21:53:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 --2- 172.21.15.64:0/1188302884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac102980 0x7f1dac1092a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 --2- 172.21.15.64:0/1188302884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dac100e70 0x7f1dac101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 --2- 172.21.15.64:0/1188302884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dac101fc0 0x7f1dac102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 -- 172.21.15.64:0/1188302884 >> 172.21.15.64:0/1188302884 conn(0x7f1dac0fc400 msgr2=0x7f1dac0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:14.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 -- 172.21.15.64:0/1188302884 shutdown_connections 2024-05-08T21:53:14.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.449+0000 7f1db0c64700 1 -- 172.21.15.64:0/1188302884 wait complete. 2024-05-08T21:53:14.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 Processor -- start 2024-05-08T21:53:14.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 -- start start 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 0x7f1dac196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dac101fc0 0x7f1dac197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dac102980 0x7f1dac19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19ea40 con 0x7f1dac100e70 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19eb80 con 0x7f1dac102980 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1db0c64700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19ed10 con 0x7f1dac101fc0 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1daa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 0x7f1dac196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1daa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 0x7f1dac196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38140/0 (socket says 172.21.15.64:38140) 2024-05-08T21:53:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1daa59c700 1 -- 172.21.15.64:0/3810713760 learned_addr learned my addr 172.21.15.64:0/3810713760 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1daad9d700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dac102980 0x7f1dac19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.450+0000 7f1da9d9b700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dac101fc0 0x7f1dac197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1daa59c700 1 -- 172.21.15.64:0/3810713760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dac101fc0 msgr2=0x7f1dac197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1daa59c700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dac101fc0 0x7f1dac197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1daa59c700 1 -- 172.21.15.64:0/3810713760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dac102980 msgr2=0x7f1dac19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1daa59c700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dac102980 0x7f1dac19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1daa59c700 1 -- 172.21.15.64:0/3810713760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da4009d00 con 0x7f1dac100e70 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1daa59c700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 0x7f1dac196f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00d980 tx=0x7f1d9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c004a30 con 0x7f1dac100e70 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac19eff0 con 0x7f1dac100e70 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d9c0054e0 con 0x7f1dac100e70 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c009c90 con 0x7f1dac100e70 2024-05-08T21:53:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.451+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac19f410 con 0x7f1dac100e70 2024-05-08T21:53:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.452+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac0673f0 con 0x7f1dac100e70 2024-05-08T21:53:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.453+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1d9c004c20 con 0x7f1dac100e70 2024-05-08T21:53:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.454+0000 7f1d9b7fe700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1d940a1120 0x7f1d940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.454+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d9c0cc520 con 0x7f1dac100e70 2024-05-08T21:53:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.457+0000 7f1da9d9b700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1d940a1120 0x7f1d940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.457+0000 7f1da9d9b700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1d940a1120 0x7f1d940a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005950 tx=0x7f1da00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:14.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.458+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1d9c096150 con 0x7f1dac100e70 2024-05-08T21:53:14.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.717+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dac0673f0 con 0x7f1dac100e70 2024-05-08T21:53:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.718+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/3810713760 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8088 (secure 0 0 0) 0x7f1dac0673f0 con 0x7f1dac100e70 2024-05-08T21:53:14.719 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:14.719 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T21:53:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.720+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1d940a1120 msgr2=0x7f1d940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.720+0000 7f1db0c64700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1d940a1120 0x7f1d940a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005950 tx=0x7f1da00058e0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.720+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 msgr2=0x7f1dac196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.720+0000 7f1db0c64700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 0x7f1dac196f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00d980 tx=0x7f1d9c00dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 shutdown_connections 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1d940a1120 0x7f1d940a35e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dac102980 0x7f1dac19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dac100e70 0x7f1dac196f10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 --2- 172.21.15.64:0/3810713760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dac101fc0 0x7f1dac197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 >> 172.21.15.64:0/3810713760 conn(0x7f1dac0fc400 msgr2=0x7f1dac105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 shutdown_connections 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:14.721+0000 7f1db0c64700 1 -- 172.21.15.64:0/3810713760 wait complete. 2024-05-08T21:53:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: Deploying daemon mds.a on smithi064 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.64:6838/3760541428,v1:172.21.15.64:6839/3760541428] up:boot 2024-05-08T21:53:15.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: fsmap 10 up:standby 2024-05-08T21:53:15.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T21:53:15.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3810713760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:15.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: Deploying daemon mds.a on smithi064 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.64:6838/3760541428,v1:172.21.15.64:6839/3760541428] up:boot 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: fsmap 10 up:standby 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T21:53:15.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:15 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3810713760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:15.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:15.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:15.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:15.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:15.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: Deploying daemon mds.a on smithi064 2024-05-08T21:53:15.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:15.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.64:6838/3760541428,v1:172.21.15.64:6839/3760541428] up:boot 2024-05-08T21:53:15.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: fsmap 10 up:standby 2024-05-08T21:53:15.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T21:53:15.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:15 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3810713760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:15.574 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 21:53:15 smithi064 systemd[1]: Starting Ceph mds.a for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:16.213 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-mds[94902]: -- 172.21.15.64:0/2603370923 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55df6eef81a0 con 0x55df6eece800 2024-05-08T21:53:16.213 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 21:53:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a[94879]: starting mds.a at 2024-05-08T21:53:16.213 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 21:53:15 smithi064 bash[94865]: 68f5e40912262dddb13e6a1f99c40a67d0aa191c5ea72599a96d6c20205b01a1 2024-05-08T21:53:16.213 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 21:53:15 smithi064 systemd[1]: Started Ceph mds.a for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:16.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 -- 172.21.15.64:0/383585775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec102980 msgr2=0x7f97ec1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/383585775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec102980 0x7f97ec1092a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97e400b0a0 tx=0x7f97e400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 -- 172.21.15.64:0/383585775 shutdown_connections 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/383585775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec102980 0x7f97ec1092a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/383585775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97ec100e70 0x7f97ec101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/383585775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97ec101fc0 0x7f97ec102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 -- 172.21.15.64:0/383585775 >> 172.21.15.64:0/383585775 conn(0x7f97ec0fc400 msgr2=0x7f97ec0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 -- 172.21.15.64:0/383585775 shutdown_connections 2024-05-08T21:53:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.882+0000 7f97f2a2d700 1 -- 172.21.15.64:0/383585775 wait complete. 2024-05-08T21:53:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.883+0000 7f97f2a2d700 1 Processor -- start 2024-05-08T21:53:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.883+0000 7f97f2a2d700 1 -- start start 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f2a2d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97ec100e70 0x7f97ec196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f2a2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 0x7f97ec197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f2a2d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97ec102980 0x7f97ec19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f2a2d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19ea00 con 0x7f97ec101fc0 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f2a2d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19eb40 con 0x7f97ec100e70 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f2a2d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19ecd0 con 0x7f97ec102980 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 0x7f97ec197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 0x7f97ec197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38174/0 (socket says 172.21.15.64:38174) 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 -- 172.21.15.64:0/707990044 learned_addr learned my addr 172.21.15.64:0/707990044 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97ebfff700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97ec100e70 0x7f97ec196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97f0fca700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97ec102980 0x7f97ec19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 -- 172.21.15.64:0/707990044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97ec102980 msgr2=0x7f97ec19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97ec102980 0x7f97ec19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 -- 172.21.15.64:0/707990044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97ec100e70 msgr2=0x7f97ec196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97ec100e70 0x7f97ec196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 -- 172.21.15.64:0/707990044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4009d00 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97eb7fe700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 0x7f97ec197410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e9a0 tx=0x7f97d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.884+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4009790 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.885+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d4010430 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.885+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec19efb0 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.885+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4018680 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.885+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec19f430 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.886+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec108260 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.887+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f97d40105a0 con 0x7f97ec101fc0 2024-05-08T21:53:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.887+0000 7f97e97fa700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97d80a1120 0x7f97d80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.887+0000 7f97ebfff700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97d80a1120 0x7f97d80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.887+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f97d4014070 con 0x7f97ec101fc0 2024-05-08T21:53:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.888+0000 7f97ebfff700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97d80a1120 0x7f97d80a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f97dc007c60 tx=0x7f97dc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:16.892+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97d4052d60 con 0x7f97ec101fc0 2024-05-08T21:53:17.021 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:17 smithi160 systemd[1]: Starting Ceph mds.b for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T21:53:17.021 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: Deploying daemon mds.b on smithi160 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.64:6840/1669365277,v1:172.21.15.64:6841/1669365277] up:boot 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: fsmap 11 up:standby 2024-05-08T21:53:17.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:16 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:53:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: Deploying daemon mds.b on smithi160 2024-05-08T21:53:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.64:6840/1669365277,v1:172.21.15.64:6841/1669365277] up:boot 2024-05-08T21:53:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: fsmap 11 up:standby 2024-05-08T21:53:17.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:16 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:53:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.148+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97ec067380 con 0x7f97ec101fc0 2024-05-08T21:53:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.149+0000 7f97e97fa700 1 -- 172.21.15.64:0/707990044 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8846 (secure 0 0 0) 0x7f97d40961a0 con 0x7f97ec101fc0 2024-05-08T21:53:17.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:17.150 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T21:53:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97d80a1120 msgr2=0x7f97d80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97d80a1120 0x7f97d80a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f97dc007c60 tx=0x7f97dc0073d0 comp rx=0 tx=0).stop 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 msgr2=0x7f97ec197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 0x7f97ec197410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e9a0 tx=0x7f97d400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 shutdown_connections 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97d80a1120 0x7f97d80a35e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97ec100e70 0x7f97ec196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97ec102980 0x7f97ec19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 --2- 172.21.15.64:0/707990044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97ec101fc0 0x7f97ec197410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 >> 172.21.15.64:0/707990044 conn(0x7f97ec0fc400 msgr2=0x7f97ec1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 shutdown_connections 2024-05-08T21:53:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:17.151+0000 7f97f2a2d700 1 -- 172.21.15.64:0/707990044 wait complete. 2024-05-08T21:53:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: Deploying daemon mds.b on smithi160 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.64:6840/1669365277,v1:172.21.15.64:6841/1669365277] up:boot 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: fsmap 11 up:standby 2024-05-08T21:53:17.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:16 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:53:17.794 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mds[82991]: -- 172.21.15.160:0/4260279045 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56298d6f41a0 con 0x56298d6ca800 2024-05-08T21:53:17.794 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: starting mds.b at 2024-05-08T21:53:17.794 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:17 smithi160 bash[82953]: 54d0438d1bf84b4e6546753d17d34ad0da8f8104a3badeba4f0de963518f94d0 2024-05-08T21:53:17.794 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:17 smithi160 systemd[1]: Started Ceph mds.b for 71e641de-0d84-11ef-bc97-c7b262605968. 2024-05-08T21:53:18.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/707990044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:17 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/707990044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:17 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:53:18.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/707990044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:17 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:53:19.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:19.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 -- 172.21.15.64:0/3195203709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 msgr2=0x7f8fd0103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/3195203709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0103cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009990 tx=0x7f8fc0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 -- 172.21.15.64:0/3195203709 shutdown_connections 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/3195203709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd0106b40 0x7f8fd0108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/3195203709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0103cd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/3195203709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0104210 0x7f8fd0106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 -- 172.21.15.64:0/3195203709 >> 172.21.15.64:0/3195203709 conn(0x7f8fd00fb2f0 msgr2=0x7f8fd00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.301+0000 7f8fd64b8700 1 -- 172.21.15.64:0/3195203709 shutdown_connections 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 -- 172.21.15.64:0/3195203709 wait complete. 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 Processor -- start 2024-05-08T21:53:19.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 -- start start 2024-05-08T21:53:19.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd0104210 0x7f8fd00ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0106b40 0x7f8fd00ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0100570 con 0x7f8fd01018e0 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01006e0 con 0x7f8fd0104210 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.302+0000 7f8fd64b8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01a0e20 con 0x7f8fd0106b40 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38190/0 (socket says 172.21.15.64:38190) 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 -- 172.21.15.64:0/1516234371 learned_addr learned my addr 172.21.15.64:0/1516234371 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:19.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fd4a55700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0106b40 0x7f8fd00ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcf7fe700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd0104210 0x7f8fd00ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 -- 172.21.15.64:0/1516234371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0106b40 msgr2=0x7f8fd00ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0106b40 0x7f8fd00ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 -- 172.21.15.64:0/1516234371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd0104210 msgr2=0x7f8fd00ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd0104210 0x7f8fd00ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 -- 172.21.15.64:0/1516234371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00095f0 con 0x7f8fd01018e0 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fd4a55700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0106b40 0x7f8fd00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcffff700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0101200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b680 tx=0x7f8fc0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001d070 con 0x7f8fd01018e0 2024-05-08T21:53:19.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc0022470 con 0x7f8fd01018e0 2024-05-08T21:53:19.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd01a1050 con 0x7f8fd01018e0 2024-05-08T21:53:19.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0017650 con 0x7f8fd01018e0 2024-05-08T21:53:19.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.303+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01a1410 con 0x7f8fd01018e0 2024-05-08T21:53:19.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.305+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd00fce70 con 0x7f8fd01018e0 2024-05-08T21:53:19.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.306+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8fc0022870 con 0x7f8fd01018e0 2024-05-08T21:53:19.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.306+0000 7f8fcd7fa700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8fb80a1120 0x7f8fb80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.306+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fc00cd970 con 0x7f8fd01018e0 2024-05-08T21:53:19.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.306+0000 7f8fcf7fe700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8fb80a1120 0x7f8fb80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.307+0000 7f8fcf7fe700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8fb80a1120 0x7f8fb80a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009bf0 tx=0x7f8fc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:19.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.311+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8fc0097630 con 0x7f8fd01018e0 2024-05-08T21:53:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] up:boot 2024-05-08T21:53:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: fsmap 12 up:standby 2024-05-08T21:53:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:53:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:19 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] up:boot 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: fsmap 12 up:standby 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:19 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] up:boot 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: fsmap 12 up:standby 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:19 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:19.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.574+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fd0067c10 con 0x7f8fd01018e0 2024-05-08T21:53:19.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.575+0000 7f8fcd7fa700 1 -- 172.21.15.64:0/1516234371 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9607 (secure 0 0 0) 0x7f8fc009ac50 con 0x7f8fd01018e0 2024-05-08T21:53:19.577 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:19.577 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T21:53:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8fb80a1120 msgr2=0x7f8fb80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8fb80a1120 0x7f8fb80a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009bf0 tx=0x7f8fc4009260 comp rx=0 tx=0).stop 2024-05-08T21:53:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 msgr2=0x7f8fd0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0101200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b680 tx=0x7f8fc0009580 comp rx=0 tx=0).stop 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 shutdown_connections 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8fb80a1120 0x7f8fb80a35e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fd0106b40 0x7f8fd00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd01018e0 0x7f8fd0101200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 --2- 172.21.15.64:0/1516234371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd0104210 0x7f8fd00ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 >> 172.21.15.64:0/1516234371 conn(0x7f8fd00fb2f0 msgr2=0x7f8fd0103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 shutdown_connections 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.579+0000 7f8fd64b8700 1 -- 172.21.15.64:0/1516234371 wait complete. 2024-05-08T21:53:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-05-08T21:53:19.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 -- 172.21.15.64:0/566860955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 msgr2=0x7fe4201028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 --2- 172.21.15.64:0/566860955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe4201028d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe410009990 tx=0x7fe410009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 -- 172.21.15.64:0/566860955 shutdown_connections 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 --2- 172.21.15.64:0/566860955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200ff8c0 0x7fe420109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 --2- 172.21.15.64:0/566860955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe4201028d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 --2- 172.21.15.64:0/566860955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4200fef20 0x7fe4200ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 -- 172.21.15.64:0/566860955 >> 172.21.15.64:0/566860955 conn(0x7fe4200faa80 msgr2=0x7fe4200fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 -- 172.21.15.64:0/566860955 shutdown_connections 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 -- 172.21.15.64:0/566860955 wait complete. 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.754+0000 7fe4254a6700 1 Processor -- start 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 -- start start 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200fef20 0x7fe420101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4200ff8c0 0x7fe4201001f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe420100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420100ed0 con 0x7fe4201024b0 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420101040 con 0x7fe4200fef20 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe4254a6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201a0df0 con 0x7fe4200ff8c0 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe420100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe420100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38208/0 (socket says 172.21.15.64:38208) 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 -- 172.21.15.64:0/1708474750 learned_addr learned my addr 172.21.15.64:0/1708474750 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41effd700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200fef20 0x7fe420101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41e7fc700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4200ff8c0 0x7fe4201001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 -- 172.21.15.64:0/1708474750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4200ff8c0 msgr2=0x7fe4201001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4200ff8c0 0x7fe4201001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 -- 172.21.15.64:0/1708474750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200fef20 msgr2=0x7fe420101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200fef20 0x7fe420101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41f7fe700 1 -- 172.21.15.64:0/1708474750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100095f0 con 0x7fe4201024b0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.755+0000 7fe41effd700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200fef20 0x7fe420101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.756+0000 7fe41f7fe700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe420100730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe41800ef30 tx=0x7fe41800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.756+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418018070 con 0x7fe4201024b0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.756+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4201a1020 con 0x7fe4201024b0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.756+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe41800f040 con 0x7fe4201024b0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.756+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800cca0 con 0x7fe4201024b0 2024-05-08T21:53:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.756+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4201a1530 con 0x7fe4201024b0 2024-05-08T21:53:19.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.757+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4201084d0 con 0x7fe4201024b0 2024-05-08T21:53:19.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.759+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe418003a00 con 0x7fe4201024b0 2024-05-08T21:53:19.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.759+0000 7fe407fff700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe4080a1120 0x7fe4080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:19.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.759+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4180cbf80 con 0x7fe4201024b0 2024-05-08T21:53:19.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.759+0000 7fe41effd700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe4080a1120 0x7fe4080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:19.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.760+0000 7fe41effd700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe4080a1120 0x7fe4080a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b680 tx=0x7fe410005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:19.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.764+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe4180ffd60 con 0x7fe4201024b0 2024-05-08T21:53:19.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.984+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe420068060 con 0x7fe4201024b0 2024-05-08T21:53:19.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.986+0000 7fe407fff700 1 -- 172.21.15.64:0/1708474750 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18422 (secure 0 0 0) 0x7fe4180b7e60 con 0x7fe4201024b0 2024-05-08T21:53:19.988 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:19.988 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:52:49.704785+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe4080a1120 msgr2=0x7fe4080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe4080a1120 0x7fe4080a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b680 tx=0x7fe410005fd0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 msgr2=0x7fe420100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe420100730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe41800ef30 tx=0x7fe41800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 shutdown_connections 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe4080a1120 0x7fe4080a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4200fef20 0x7fe420101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4201024b0 0x7fe420100730 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 --2- 172.21.15.64:0/1708474750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4200ff8c0 0x7fe4201001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:19.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 >> 172.21.15.64:0/1708474750 conn(0x7fe4200faa80 msgr2=0x7fe420105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:19.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 shutdown_connections 2024-05-08T21:53:19.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:19.988+0000 7fe4254a6700 1 -- 172.21.15.64:0/1708474750 wait complete. 2024-05-08T21:53:20.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:53:20.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:53:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 -- 172.21.15.64:0/1589568126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c81001b0 msgr2=0x7f71c8100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 --2- 172.21.15.64:0/1589568126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c81001b0 0x7f71c8100630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f71b80099c0 tx=0x7f71b8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 -- 172.21.15.64:0/1589568126 shutdown_connections 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 --2- 172.21.15.64:0/1589568126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c8100b70 0x7f71c8100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 --2- 172.21.15.64:0/1589568126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80ff850 0x7f71c80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 --2- 172.21.15.64:0/1589568126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c81001b0 0x7f71c8100630 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 -- 172.21.15.64:0/1589568126 >> 172.21.15.64:0/1589568126 conn(0x7f71c80fb3f0 msgr2=0x7f71c80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 -- 172.21.15.64:0/1589568126 shutdown_connections 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.144+0000 7f71d0500700 1 -- 172.21.15.64:0/1589568126 wait complete. 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 Processor -- start 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 -- start start 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 0x7f71c81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c81001b0 0x7f71c8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c8100b70 0x7f71c81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81a0c20 con 0x7f71c80ff850 2024-05-08T21:53:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81a0d60 con 0x7f71c8100b70 2024-05-08T21:53:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71d0500700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81a0ef0 con 0x7f71c81001b0 2024-05-08T21:53:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 0x7f71c81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 0x7f71c81990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38226/0 (socket says 172.21.15.64:38226) 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 -- 172.21.15.64:0/3669443101 learned_addr learned my addr 172.21.15.64:0/3669443101 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71cda9b700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c81001b0 0x7f71c8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71cea9d700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c8100b70 0x7f71c81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 -- 172.21.15.64:0/3669443101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c81001b0 msgr2=0x7f71c8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c81001b0 0x7f71c8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 -- 172.21.15.64:0/3669443101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c8100b70 msgr2=0x7f71c81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c8100b70 0x7f71c81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71ce29c700 1 -- 172.21.15.64:0/3669443101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80095f0 con 0x7f71c80ff850 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.145+0000 7f71cda9b700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c81001b0 0x7f71c8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.146+0000 7f71ce29c700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 0x7f71c81990f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e9a0 tx=0x7f71c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.146+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4009790 con 0x7f71c80ff850 2024-05-08T21:53:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.146+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c81a11d0 con 0x7f71c80ff850 2024-05-08T21:53:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.146+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c81a1730 con 0x7f71c80ff850 2024-05-08T21:53:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.146+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71c4010430 con 0x7f71c80ff850 2024-05-08T21:53:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.146+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4018680 con 0x7f71c80ff850 2024-05-08T21:53:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.147+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac005320 con 0x7f71c80ff850 2024-05-08T21:53:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.149+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f71c40105a0 con 0x7f71c80ff850 2024-05-08T21:53:20.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.150+0000 7f71bf7fe700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f71b40a11a0 0x7f71b40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.150+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71c4014070 con 0x7f71c80ff850 2024-05-08T21:53:20.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.150+0000 7f71cda9b700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f71b40a11a0 0x7f71b40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.150+0000 7f71cda9b700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f71b40a11a0 0x7f71b40a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71b8003730 tx=0x7f71b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:20.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.154+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f71c4096ad0 con 0x7f71c80ff850 2024-05-08T21:53:20.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.380+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71ac005b50 con 0x7f71c80ff850 2024-05-08T21:53:20.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.380+0000 7f71bf7fe700 1 -- 172.21.15.64:0/3669443101 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71c409a0f0 con 0x7f71c80ff850 2024-05-08T21:53:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:53:20.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.382+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f71b40a11a0 msgr2=0x7f71b40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.382+0000 7f71d0500700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f71b40a11a0 0x7f71b40a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71b8003730 tx=0x7f71b801a040 comp rx=0 tx=0).stop 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.382+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 msgr2=0x7f71c81990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.382+0000 7f71d0500700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 0x7f71c81990f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e9a0 tx=0x7f71c400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 shutdown_connections 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f71b40a11a0 0x7f71b40a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c8100b70 0x7f71c81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71c80ff850 0x7f71c81990f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 --2- 172.21.15.64:0/3669443101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c81001b0 0x7f71c8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 >> 172.21.15.64:0/3669443101 conn(0x7f71c80fb3f0 msgr2=0x7f71c80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 shutdown_connections 2024-05-08T21:53:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.383+0000 7f71d0500700 1 -- 172.21.15.64:0/3669443101 wait complete. 2024-05-08T21:53:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:53:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:20 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1516234371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:20 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1708474750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:53:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:20 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1516234371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:20 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1708474750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T21:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 -- 172.21.15.64:0/2167182540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa918100f50 msgr2=0x7fa9181013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 --2- 172.21.15.64:0/2167182540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa918100f50 0x7fa9181013b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0099c0 tx=0x7fa90c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 -- 172.21.15.64:0/2167182540 shutdown_connections 2024-05-08T21:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 --2- 172.21.15.64:0/2167182540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9181018f0 0x7fa9181081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 --2- 172.21.15.64:0/2167182540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9180ffd60 0x7fa918100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 --2- 172.21.15.64:0/2167182540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa918100f50 0x7fa9181013b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 -- 172.21.15.64:0/2167182540 >> 172.21.15.64:0/2167182540 conn(0x7fa9180fb2f0 msgr2=0x7fa9180fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 -- 172.21.15.64:0/2167182540 shutdown_connections 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.537+0000 7fa91e911700 1 -- 172.21.15.64:0/2167182540 wait complete. 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 Processor -- start 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 -- start start 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 0x7fa918103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa918100f50 0x7fa918102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9181018f0 0x7fa918102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918103010 con 0x7fa9180ffd60 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918103180 con 0x7fa918100f50 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa91e911700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181a0e40 con 0x7fa9181018f0 2024-05-08T21:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa917fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 0x7fa918103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa917fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 0x7fa918103ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38252/0 (socket says 172.21.15.64:38252) 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa917fff700 1 -- 172.21.15.64:0/3169719254 learned_addr learned my addr 172.21.15.64:0/3169719254 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.538+0000 7fa9177fe700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa918100f50 0x7fa918102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa91ceae700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9181018f0 0x7fa918102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa917fff700 1 -- 172.21.15.64:0/3169719254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9181018f0 msgr2=0x7fa918102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa917fff700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9181018f0 0x7fa918102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa917fff700 1 -- 172.21.15.64:0/3169719254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa918100f50 msgr2=0x7fa918102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa917fff700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa918100f50 0x7fa918102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa917fff700 1 -- 172.21.15.64:0/3169719254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0095f0 con 0x7fa9180ffd60 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa9177fe700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa918100f50 0x7fa918102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa917fff700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 0x7fa918103ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa90800d980 tx=0x7fa90800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908004020 con 0x7fa9180ffd60 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90800f690 con 0x7fa9180ffd60 2024-05-08T21:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9181a10d0 con 0x7fa9180ffd60 2024-05-08T21:53:20.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908003a60 con 0x7fa9180ffd60 2024-05-08T21:53:20.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.539+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9181a15e0 con 0x7fa9180ffd60 2024-05-08T21:53:20.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.540+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa918067c10 con 0x7fa9180ffd60 2024-05-08T21:53:20.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.542+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa90800fb00 con 0x7fa9180ffd60 2024-05-08T21:53:20.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.542+0000 7fa9157fa700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9000a1120 0x7fa9000a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.542+0000 7fa9177fe700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9000a1120 0x7fa9000a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.542+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9080cbff0 con 0x7fa9180ffd60 2024-05-08T21:53:20.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.543+0000 7fa9177fe700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9000a1120 0x7fa9000a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009990 tx=0x7fa90c005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:53:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:20.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:20 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1516234371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:20 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1708474750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:20.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.547+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa908095c20 con 0x7fa9180ffd60 2024-05-08T21:53:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.777+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa91810b570 con 0x7fa9180ffd60 2024-05-08T21:53:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.778+0000 7fa9157fa700 1 -- 172.21.15.64:0/3169719254 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa908099240 con 0x7fa9180ffd60 2024-05-08T21:53:20.779 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:20.779 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-05-08T21:53:20.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T21:53:20.780 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-05-08T21:53:20.780 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-05-08T21:53:20.780 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T21:53:20.780 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9000a1120 msgr2=0x7fa9000a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9000a1120 0x7fa9000a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009990 tx=0x7fa90c005c20 comp rx=0 tx=0).stop 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 msgr2=0x7fa918103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 0x7fa918103ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa90800d980 tx=0x7fa90800dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 shutdown_connections 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9000a1120 0x7fa9000a35e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9181018f0 0x7fa918102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9180ffd60 0x7fa918103ca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 --2- 172.21.15.64:0/3169719254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa918100f50 0x7fa918102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 >> 172.21.15.64:0/3169719254 conn(0x7fa9180fb2f0 msgr2=0x7fa918068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 shutdown_connections 2024-05-08T21:53:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.780+0000 7fa91e911700 1 -- 172.21.15.64:0/3169719254 wait complete. 2024-05-08T21:53:20.794 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T21:53:20.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 -- 172.21.15.64:0/2592642392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 msgr2=0x7fadec101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 --2- 172.21.15.64:0/2592642392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec101430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faddc009990 tx=0x7faddc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 -- 172.21.15.64:0/2592642392 shutdown_connections 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 --2- 172.21.15.64:0/2592642392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadec0ffe60 0x7fadec100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 --2- 172.21.15.64:0/2592642392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadec101970 0x7fadec108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 --2- 172.21.15.64:0/2592642392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec101430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 -- 172.21.15.64:0/2592642392 >> 172.21.15.64:0/2592642392 conn(0x7fadec0fb3f0 msgr2=0x7fadec0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 -- 172.21.15.64:0/2592642392 shutdown_connections 2024-05-08T21:53:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 -- 172.21.15.64:0/2592642392 wait complete. 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.924+0000 7fadf140b700 1 Processor -- start 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 -- start start 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadec0ffe60 0x7fadec196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadec101970 0x7fadec19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec19ea10 con 0x7fadec100fb0 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec19eb50 con 0x7fadec0ffe60 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadf140b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec19ece0 con 0x7fadec101970 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38282/0 (socket says 172.21.15.64:38282) 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 -- 172.21.15.64:0/4029533535 learned_addr learned my addr 172.21.15.64:0/4029533535 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadeaffd700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadec0ffe60 0x7fadec196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 -- 172.21.15.64:0/4029533535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadec101970 msgr2=0x7fadec19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadeb7fe700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadec101970 0x7fadec19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadec101970 0x7fadec19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 -- 172.21.15.64:0/4029533535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadec0ffe60 msgr2=0x7fadec196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadec0ffe60 0x7fadec196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 -- 172.21.15.64:0/4029533535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0095f0 con 0x7fadec100fb0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.925+0000 7fadea7fc700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec197420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faddc004760 tx=0x7faddc004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.926+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc01d070 con 0x7fadec100fb0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.926+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec19ef60 con 0x7fadec100fb0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.926+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc004830 con 0x7fadec100fb0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.926+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0054b0 con 0x7fadec100fb0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.926+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec19f320 con 0x7fadec100fb0 2024-05-08T21:53:20.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.927+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec067cf0 con 0x7fadec100fb0 2024-05-08T21:53:20.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.928+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7faddc004c10 con 0x7fadec100fb0 2024-05-08T21:53:20.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.928+0000 7fadd3fff700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fadd40a1120 0x7fadd40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:20.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.928+0000 7fadeaffd700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fadd40a1120 0x7fadd40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:20.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.928+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faddc0cd730 con 0x7fadec100fb0 2024-05-08T21:53:20.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.929+0000 7fadeaffd700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fadd40a1120 0x7fadd40a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fade0009c60 tx=0x7fade0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:20.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:20.932+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faddc0973f0 con 0x7fadec100fb0 2024-05-08T21:53:21.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:21.145+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fadec10b660 con 0x7fadec100fb0 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3669443101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:21.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:53:21.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:21 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3169719254' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3669443101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:21.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:53:21.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:21 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3169719254' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T21:53:21.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3669443101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:53:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:21 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3169719254' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T21:53:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.120+0000 7fadd3fff700 1 -- 172.21.15.64:0/4029533535 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7faddc0049a0 con 0x7fadec100fb0 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.125+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fadd40a1120 msgr2=0x7fadd40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.125+0000 7fadf140b700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fadd40a1120 0x7fadd40a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fade0009c60 tx=0x7fade0009260 comp rx=0 tx=0).stop 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.125+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 msgr2=0x7fadec197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.125+0000 7fadf140b700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec197420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faddc004760 tx=0x7faddc004790 comp rx=0 tx=0).stop 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 shutdown_connections 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fadd40a1120 0x7fadd40a35e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadec0ffe60 0x7fadec196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadec101970 0x7fadec19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 --2- 172.21.15.64:0/4029533535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadec100fb0 0x7fadec197420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 >> 172.21.15.64:0/4029533535 conn(0x7fadec0fb3f0 msgr2=0x7fadec067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 shutdown_connections 2024-05-08T21:53:22.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.126+0000 7fadf140b700 1 -- 172.21.15.64:0/4029533535 wait complete. 2024-05-08T21:53:22.129 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-05-08T21:53:22.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T21:53:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 -- 172.21.15.64:0/3951301257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47380ffe70 msgr2=0x7f4738100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 --2- 172.21.15.64:0/3951301257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47380ffe70 0x7f4738100290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4728009990 tx=0x7f4728009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 -- 172.21.15.64:0/3951301257 shutdown_connections 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 --2- 172.21.15.64:0/3951301257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4738101980 0x7f47381082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 --2- 172.21.15.64:0/3951301257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47380ffe70 0x7f4738100290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 --2- 172.21.15.64:0/3951301257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738100fc0 0x7f4738101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 -- 172.21.15.64:0/3951301257 >> 172.21.15.64:0/3951301257 conn(0x7f47380fb400 msgr2=0x7f47380fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 -- 172.21.15.64:0/3951301257 shutdown_connections 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 -- 172.21.15.64:0/3951301257 wait complete. 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 Processor -- start 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 -- start start 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.283+0000 7f473f910700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380ffe70 0x7f4738069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738100fc0 0x7f473819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 0x7f473819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473819ea20 con 0x7f4738101980 2024-05-08T21:53:22.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473819eb90 con 0x7f47380ffe70 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473819ed50 con 0x7f4738100fc0 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 0x7f473819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 0x7f473819e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38292/0 (socket says 172.21.15.64:38292) 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 -- 172.21.15.64:0/3173363152 learned_addr learned my addr 172.21.15.64:0/3173363152 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473d6ac700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380ffe70 0x7f4738069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473ceab700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738100fc0 0x7f473819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 -- 172.21.15.64:0/3173363152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738100fc0 msgr2=0x7f473819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738100fc0 0x7f473819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 -- 172.21.15.64:0/3173363152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380ffe70 msgr2=0x7f4738069f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380ffe70 0x7f4738069f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 -- 172.21.15.64:0/3173363152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280095f0 con 0x7f4738101980 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473dead700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 0x7f473819e230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f473000ada0 tx=0x7f473000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000f070 con 0x7f4738101980 2024-05-08T21:53:22.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473819efd0 con 0x7f4738101980 2024-05-08T21:53:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4730007b30 con 0x7f4738101980 2024-05-08T21:53:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730005440 con 0x7f4738101980 2024-05-08T21:53:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.284+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473819f400 con 0x7f4738101980 2024-05-08T21:53:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.285+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738107380 con 0x7f4738101980 2024-05-08T21:53:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.286+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4730004ad0 con 0x7f4738101980 2024-05-08T21:53:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.287+0000 7f472e7fc700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f47240a1120 0x7f47240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.287+0000 7f473d6ac700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f47240a1120 0x7f47240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.287+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f47300cbcc0 con 0x7f4738101980 2024-05-08T21:53:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.288+0000 7f473d6ac700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f47240a1120 0x7f47240a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f472800b5c0 tx=0x7f4728005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.292+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4730095810 con 0x7f4738101980 2024-05-08T21:53:22.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:22 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/4029533535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:53:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:22 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/4029533535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:53:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:22.501+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4738068140 con 0x7f4738101980 2024-05-08T21:53:22.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:22 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/4029533535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:53:23.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.128+0000 7f472e7fc700 1 -- 172.21.15.64:0/3173363152 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4730098e30 con 0x7f4738101980 2024-05-08T21:53:23.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.136+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f47240a1120 msgr2=0x7f47240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.136+0000 7f473f910700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f47240a1120 0x7f47240a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f472800b5c0 tx=0x7f4728005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.136+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 msgr2=0x7f473819e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.136+0000 7f473f910700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 0x7f473819e230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f473000ada0 tx=0x7f473000c330 comp rx=0 tx=0).stop 2024-05-08T21:53:23.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.139+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 shutdown_connections 2024-05-08T21:53:23.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.139+0000 7f473f910700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f47240a1120 0x7f47240a35e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.139+0000 7f473f910700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380ffe70 0x7f4738069f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.139+0000 7f473f910700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4738101980 0x7f473819e230 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.139+0000 7f473f910700 1 --2- 172.21.15.64:0/3173363152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738100fc0 0x7f473819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.139+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 >> 172.21.15.64:0/3173363152 conn(0x7f47380fb400 msgr2=0x7f47381044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:23.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.143+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 shutdown_connections 2024-05-08T21:53:23.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.143+0000 7f473f910700 1 -- 172.21.15.64:0/3173363152 wait complete. 2024-05-08T21:53:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-05-08T21:53:23.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T21:53:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 -- 172.21.15.64:0/2553556291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 msgr2=0x7f9e701082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 --2- 172.21.15.64:0/2553556291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e701082b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b0a0 tx=0x7f9e6800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 -- 172.21.15.64:0/2553556291 shutdown_connections 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 --2- 172.21.15.64:0/2553556291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e701082b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 --2- 172.21.15.64:0/2553556291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e700ffe80 0x7f9e701002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 --2- 172.21.15.64:0/2553556291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e70100fd0 0x7f9e70101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 -- 172.21.15.64:0/2553556291 >> 172.21.15.64:0/2553556291 conn(0x7f9e700fb3f0 msgr2=0x7f9e700fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 -- 172.21.15.64:0/2553556291 shutdown_connections 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.289+0000 7f9e763b7700 1 -- 172.21.15.64:0/2553556291 wait complete. 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 Processor -- start 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 -- start start 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e700ffe80 0x7f9e70069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e70100fd0 0x7f9e7019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e7019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019eaf0 con 0x7f9e70101990 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019ec60 con 0x7f9e70100fd0 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e763b7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019ee20 con 0x7f9e700ffe80 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e74954700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e7019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e74954700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e7019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46508/0 (socket says 172.21.15.64:46508) 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.290+0000 7f9e74954700 1 -- 172.21.15.64:0/914257923 learned_addr learned my addr 172.21.15.64:0/914257923 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e6f7fe700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e70100fd0 0x7f9e7019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e6ffff700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e700ffe80 0x7f9e70069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e74954700 1 -- 172.21.15.64:0/914257923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e700ffe80 msgr2=0x7f9e70069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e74954700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e700ffe80 0x7f9e70069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e74954700 1 -- 172.21.15.64:0/914257923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e70100fd0 msgr2=0x7f9e7019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e74954700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e70100fd0 0x7f9e7019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e74954700 1 -- 172.21.15.64:0/914257923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68009d00 con 0x7f9e70101990 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e74954700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e7019e350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006070 tx=0x7f9e68004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800e040 con 0x7f9e70101990 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7019f0a0 con 0x7f9e70101990 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e6801db50 con 0x7f9e70101990 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6801bc00 con 0x7f9e70101990 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.291+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7019f570 con 0x7f9e70101990 2024-05-08T21:53:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.292+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e70107220 con 0x7f9e70101990 2024-05-08T21:53:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.294+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9e6801dcc0 con 0x7f9e70101990 2024-05-08T21:53:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.294+0000 7f9e6d7fa700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9e580a1120 0x7f9e580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.294+0000 7f9e6ffff700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9e580a1120 0x7f9e580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.294+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f9e680ce950 con 0x7f9e70101990 2024-05-08T21:53:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.295+0000 7f9e6ffff700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9e580a1120 0x7f9e580a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.298+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e68098320 con 0x7f9e70101990 2024-05-08T21:53:23.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:23 smithi186 ceph-mon[46619]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:23.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:23 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/4029533535' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T21:53:23.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:23 smithi186 ceph-mon[46619]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T21:53:23.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:23 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3173363152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T21:53:23.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:23 smithi064 ceph-mon[42500]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:23.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:23 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/4029533535' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T21:53:23.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:23 smithi064 ceph-mon[42500]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T21:53:23.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:23 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3173363152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T21:53:23.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:23 smithi160 ceph-mon[46962]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:53:23.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:23 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/4029533535' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T21:53:23.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:23 smithi160 ceph-mon[46962]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T21:53:23.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:23 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3173363152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T21:53:23.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.553+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9e70067460 con 0x7f9e70101990 2024-05-08T21:53:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.559+0000 7f9e6d7fa700 1 -- 172.21.15.64:0/914257923 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9e6809b940 con 0x7f9e70101990 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9e580a1120 msgr2=0x7f9e580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9e580a1120 0x7f9e580a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e60008040 comp rx=0 tx=0).stop 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 msgr2=0x7f9e7019e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e7019e350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006070 tx=0x7f9e68004d00 comp rx=0 tx=0).stop 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 shutdown_connections 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9e580a1120 0x7f9e580a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e70101990 0x7f9e7019e350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e700ffe80 0x7f9e70069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 --2- 172.21.15.64:0/914257923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e70100fd0 0x7f9e7019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.565+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 >> 172.21.15.64:0/914257923 conn(0x7f9e700fb3f0 msgr2=0x7f9e70104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.568+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 shutdown_connections 2024-05-08T21:53:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.568+0000 7f9e763b7700 1 -- 172.21.15.64:0/914257923 wait complete. 2024-05-08T21:53:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T21:53:23.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 -- 172.21.15.64:0/3179897482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b701018e0 msgr2=0x7f7b70103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 --2- 172.21.15.64:0/3179897482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b701018e0 0x7f7b70103cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009990 tx=0x7f7b64009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 -- 172.21.15.64:0/3179897482 shutdown_connections 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 --2- 172.21.15.64:0/3179897482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b70106b40 0x7f7b70108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 --2- 172.21.15.64:0/3179897482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b701018e0 0x7f7b70103cd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 --2- 172.21.15.64:0/3179897482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b70104210 0x7f7b70106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 -- 172.21.15.64:0/3179897482 >> 172.21.15.64:0/3179897482 conn(0x7f7b700fb2f0 msgr2=0x7f7b700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 -- 172.21.15.64:0/3179897482 shutdown_connections 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 -- 172.21.15.64:0/3179897482 wait complete. 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 Processor -- start 2024-05-08T21:53:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.716+0000 7f7b76b52700 1 -- start start 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b701018e0 0x7f7b70196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 0x7f7b70197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b70106b40 0x7f7b7019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7019e820 con 0x7f7b70104210 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7019e990 con 0x7f7b701018e0 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7019eb50 con 0x7f7b70106b40 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 0x7f7b70197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 0x7f7b70197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46530/0 (socket says 172.21.15.64:46530) 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 -- 172.21.15.64:0/2506046653 learned_addr learned my addr 172.21.15.64:0/2506046653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b748ee700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b701018e0 0x7f7b70196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b750ef700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b70106b40 0x7f7b7019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 -- 172.21.15.64:0/2506046653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b70106b40 msgr2=0x7f7b7019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b70106b40 0x7f7b7019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 -- 172.21.15.64:0/2506046653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b701018e0 msgr2=0x7f7b70196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b701018e0 0x7f7b70196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 -- 172.21.15.64:0/2506046653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640095f0 con 0x7f7b70104210 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b748ee700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b701018e0 0x7f7b70196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6ffff700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 0x7f7b70197240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00d950 tx=0x7f7b5c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c009b40 con 0x7f7b70104210 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5c009ca0 con 0x7f7b70104210 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7019ee30 con 0x7f7b70104210 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.717+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00f620 con 0x7f7b70104210 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.718+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7019f300 con 0x7f7b70104210 2024-05-08T21:53:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.718+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b700fce70 con 0x7f7b70104210 2024-05-08T21:53:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.719+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7b5c010460 con 0x7f7b70104210 2024-05-08T21:53:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.720+0000 7f7b6dffb700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b600a11a0 0x7f7b600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.720+0000 7f7b748ee700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b600a11a0 0x7f7b600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.720+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7b5c0ccb40 con 0x7f7b70104210 2024-05-08T21:53:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.720+0000 7f7b748ee700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b600a11a0 0x7f7b600a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:23.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.724+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b5c096380 con 0x7f7b70104210 2024-05-08T21:53:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.935+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b70067c10 con 0x7f7b70104210 2024-05-08T21:53:23.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.937+0000 7f7b6dffb700 1 -- 172.21.15.64:0/2506046653 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22132 (secure 0 0 0) 0x7f7b5c07f690 con 0x7f7b70104210 2024-05-08T21:53:23.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:23.939 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":78,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:23.555191+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:53:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b600a11a0 msgr2=0x7f7b600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b600a11a0 0x7f7b600a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b64005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 msgr2=0x7f7b70197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 0x7f7b70197240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00d950 tx=0x7f7b5c00dd10 comp rx=0 tx=0).stop 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 shutdown_connections 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b600a11a0 0x7f7b600a3660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b701018e0 0x7f7b70196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b70106b40 0x7f7b7019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 --2- 172.21.15.64:0/2506046653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b70104210 0x7f7b70197240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 >> 172.21.15.64:0/2506046653 conn(0x7f7b700fb2f0 msgr2=0x7f7b70103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 shutdown_connections 2024-05-08T21:53:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:23.939+0000 7f7b76b52700 1 -- 172.21.15.64:0/2506046653 wait complete. 2024-05-08T21:53:23.955 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:53:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 -- 172.21.15.64:0/1176622298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4100e80 msgr2=0x7f06a41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 --2- 172.21.15.64:0/1176622298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4100e80 0x7f06a41012a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009990 tx=0x7f06a0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 -- 172.21.15.64:0/1176622298 shutdown_connections 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 --2- 172.21.15.64:0/1176622298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a4102990 0x7f06a41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 --2- 172.21.15.64:0/1176622298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4100e80 0x7f06a41012a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 --2- 172.21.15.64:0/1176622298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4101fd0 0x7f06a4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 -- 172.21.15.64:0/1176622298 >> 172.21.15.64:0/1176622298 conn(0x7f06a40fc410 msgr2=0x7f06a40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 -- 172.21.15.64:0/1176622298 shutdown_connections 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.080+0000 7f06abc9a700 1 -- 172.21.15.64:0/1176622298 wait complete. 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 Processor -- start 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 -- start start 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4100e80 0x7f06a4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 0x7f06a41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a4102990 0x7f06a419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419e8c0 con 0x7f06a4101fd0 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419ea00 con 0x7f06a4102990 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06abc9a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419eb90 con 0x7f06a4100e80 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 0x7f06a41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 0x7f06a41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46548/0 (socket says 172.21.15.64:46548) 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 -- 172.21.15.64:0/2299825627 learned_addr learned my addr 172.21.15.64:0/2299825627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9a36700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4100e80 0x7f06a4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06aa237700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a4102990 0x7f06a419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 -- 172.21.15.64:0/2299825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4100e80 msgr2=0x7f06a4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4100e80 0x7f06a4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 -- 172.21.15.64:0/2299825627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a4102990 msgr2=0x7f06a419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a4102990 0x7f06a419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 -- 172.21.15.64:0/2299825627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00095f0 con 0x7f06a4101fd0 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9a36700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4100e80 0x7f06a4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:24.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.081+0000 7f06a9235700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 0x7f06a41972d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f069400e9a0 tx=0x7f069400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:24.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.082+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694009790 con 0x7f06a4101fd0 2024-05-08T21:53:24.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.082+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f069400cbd0 con 0x7f06a4101fd0 2024-05-08T21:53:24.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.082+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a419ee70 con 0x7f06a4101fd0 2024-05-08T21:53:24.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.082+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694018680 con 0x7f06a4101fd0 2024-05-08T21:53:24.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.082+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a419f340 con 0x7f06a4101fd0 2024-05-08T21:53:24.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.083+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a41082c0 con 0x7f06a4101fd0 2024-05-08T21:53:24.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.084+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f069400cd40 con 0x7f06a4101fd0 2024-05-08T21:53:24.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.084+0000 7f069affd700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f06900a11a0 0x7f06900a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.084+0000 7f06a9a36700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f06900a11a0 0x7f06900a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.084+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0694014070 con 0x7f06a4101fd0 2024-05-08T21:53:24.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.085+0000 7f06a9a36700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f06900a11a0 0x7f06900a3660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f06a000b2e0 tx=0x7f06a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.089+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0694052d60 con 0x7f06a4101fd0 2024-05-08T21:53:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3173363152' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T21:53:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T21:53:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/914257923' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T21:53:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T21:53:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T21:53:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/914257923' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T21:53:24.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T21:53:24.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T21:53:24.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2506046653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:24.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.301+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06a4067c10 con 0x7f06a4101fd0 2024-05-08T21:53:24.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.302+0000 7f069affd700 1 -- 172.21.15.64:0/2299825627 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22132 (secure 0 0 0) 0x7f0694099da0 con 0x7f06a4101fd0 2024-05-08T21:53:24.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:24.304 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":78,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:23.555191+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:53:24.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f06900a11a0 msgr2=0x7f06900a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f06900a11a0 0x7f06900a3660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f06a000b2e0 tx=0x7f06a0005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 msgr2=0x7f06a41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 0x7f06a41972d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f069400e9a0 tx=0x7f069400ed60 comp rx=0 tx=0).stop 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 shutdown_connections 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f06900a11a0 0x7f06900a3660 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a4102990 0x7f06a419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a4100e80 0x7f06a4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 --2- 172.21.15.64:0/2299825627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a4101fd0 0x7f06a41972d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 >> 172.21.15.64:0/2299825627 conn(0x7f06a40fc410 msgr2=0x7f06a4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.304+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 shutdown_connections 2024-05-08T21:53:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.305+0000 7f06abc9a700 1 -- 172.21.15.64:0/2299825627 wait complete. 2024-05-08T21:53:24.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T21:53:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3173363152' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T21:53:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T21:53:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/914257923' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T21:53:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T21:53:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T21:53:24.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/914257923' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T21:53:24.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T21:53:24.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T21:53:24.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:24 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2506046653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:24.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 -- 172.21.15.64:0/4200272586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c104310 msgr2=0x7f749c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 --2- 172.21.15.64:0/4200272586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c104310 0x7f749c106700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7490009990 tx=0x7f7490009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 -- 172.21.15.64:0/4200272586 shutdown_connections 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 --2- 172.21.15.64:0/4200272586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c106c40 0x7f749c1090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 --2- 172.21.15.64:0/4200272586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c1019e0 0x7f749c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 --2- 172.21.15.64:0/4200272586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c104310 0x7f749c106700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 -- 172.21.15.64:0/4200272586 >> 172.21.15.64:0/4200272586 conn(0x7f749c0fb3f0 msgr2=0x7f749c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 -- 172.21.15.64:0/4200272586 shutdown_connections 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.446+0000 7f74a1f12700 1 -- 172.21.15.64:0/4200272586 wait complete. 2024-05-08T21:53:24.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.447+0000 7f74a1f12700 1 Processor -- start 2024-05-08T21:53:24.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.447+0000 7f74a1f12700 1 -- start start 2024-05-08T21:53:24.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 0x7f749c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c104310 0x7f749c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c106c40 0x7f749c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19ea00 con 0x7f749c1019e0 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19eb40 con 0x7f749c104310 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19ecd0 con 0x7f749c106c40 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 0x7f749c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 0x7f749c196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46578/0 (socket says 172.21.15.64:46578) 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 -- 172.21.15.64:0/2359929769 learned_addr learned my addr 172.21.15.64:0/2359929769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749bfff700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c106c40 0x7f749c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749affd700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c104310 0x7f749c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 -- 172.21.15.64:0/2359929769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c106c40 msgr2=0x7f749c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c106c40 0x7f749c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 -- 172.21.15.64:0/2359929769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c104310 msgr2=0x7f749c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c104310 0x7f749c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 -- 172.21.15.64:0/2359929769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74900095f0 con 0x7f749c1019e0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749bfff700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c106c40 0x7f749c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749affd700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c104310 0x7f749c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f749b7fe700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 0x7f749c196ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c920 tx=0x7f748c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c007b00 con 0x7f749c1019e0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c004cf0 con 0x7f749c1019e0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c19efb0 con 0x7f749c1019e0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.448+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c005610 con 0x7f749c1019e0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.449+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c19f3d0 con 0x7f749c1019e0 2024-05-08T21:53:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.449+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c0fcf70 con 0x7f749c1019e0 2024-05-08T21:53:24.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.451+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f748c004750 con 0x7f749c1019e0 2024-05-08T21:53:24.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.451+0000 7f7498ff9700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f74840a1120 0x7f74840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:24.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.451+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f748c0cc700 con 0x7f749c1019e0 2024-05-08T21:53:24.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.451+0000 7f749affd700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f74840a1120 0x7f74840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:24.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.452+0000 7f749affd700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f74840a1120 0x7f74840a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f749000b2e0 tx=0x7f7490005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.455+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f748c095fe0 con 0x7f749c1019e0 2024-05-08T21:53:24.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3173363152' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/914257923' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/914257923' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T21:53:24.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:24 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2506046653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:24.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:24.716+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f749c067cc0 con 0x7f749c1019e0 2024-05-08T21:53:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: pgmap v220: 97 pgs: 15 creating+peering, 31 active+clean, 51 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:53:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2299825627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] up:active 2024-05-08T21:53:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T21:53:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:25.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:25 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2359929769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: pgmap v220: 97 pgs: 15 creating+peering, 31 active+clean, 51 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2299825627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] up:active 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:25.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:25 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2359929769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T21:53:25.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: pgmap v220: 97 pgs: 15 creating+peering, 31 active+clean, 51 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:53:25.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2299825627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:25.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] up:active 2024-05-08T21:53:25.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:25.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T21:53:25.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:25.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:25 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2359929769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T21:53:25.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.573+0000 7f7498ff9700 1 -- 172.21.15.64:0/2359929769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f748c01d020 con 0x7f749c1019e0 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f74840a1120 msgr2=0x7f74840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f74840a1120 0x7f74840a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f749000b2e0 tx=0x7f7490005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 msgr2=0x7f749c196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 0x7f749c196ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c920 tx=0x7f748c00cce0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 shutdown_connections 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f74840a1120 0x7f74840a35e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c106c40 0x7f749c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f749c1019e0 0x7f749c196ed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.575+0000 7f74a1f12700 1 --2- 172.21.15.64:0/2359929769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f749c104310 0x7f749c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.576+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 >> 172.21.15.64:0/2359929769 conn(0x7f749c0fb3f0 msgr2=0x7f749c103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:25.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.576+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 shutdown_connections 2024-05-08T21:53:25.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.576+0000 7f74a1f12700 1 -- 172.21.15.64:0/2359929769 wait complete. 2024-05-08T21:53:25.592 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-08T21:53:25.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/3160605796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c100f20 msgr2=0x7fe29c101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/3160605796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c100f20 0x7fe29c101360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe290009990 tx=0x7fe290009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/3160605796 shutdown_connections 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/3160605796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe29c102ab0 0x7fe29c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/3160605796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c100f20 0x7fe29c101360 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/3160605796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c1020f0 0x7fe29c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/3160605796 >> 172.21.15.64:0/3160605796 conn(0x7fe29c0fc4e0 msgr2=0x7fe29c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/3160605796 shutdown_connections 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.726+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/3160605796 wait complete. 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 Processor -- start 2024-05-08T21:53:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 -- start start 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c100f20 0x7fe29c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe29c1020f0 0x7fe29c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 0x7fe29c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c19ea60 con 0x7fe29c102ab0 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c19eba0 con 0x7fe29c1020f0 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a2a9b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c19ed30 con 0x7fe29c100f20 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 0x7fe29c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 0x7fe29c19e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46602/0 (socket says 172.21.15.64:46602) 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 -- 172.21.15.64:0/514526246 learned_addr learned my addr 172.21.15.64:0/514526246 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe29bfff700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe29c1020f0 0x7fe29c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a0837700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c100f20 0x7fe29c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 -- 172.21.15.64:0/514526246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c100f20 msgr2=0x7fe29c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c100f20 0x7fe29c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 -- 172.21.15.64:0/514526246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe29c1020f0 msgr2=0x7fe29c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe29c1020f0 0x7fe29c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a1038700 1 -- 172.21.15.64:0/514526246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900095f0 con 0x7fe29c102ab0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.727+0000 7fe2a0837700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c100f20 0x7fe29c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.728+0000 7fe2a1038700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 0x7fe29c19e490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe294007ee0 tx=0x7fe29400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.728+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294017070 con 0x7fe29c102ab0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.728+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29400f040 con 0x7fe29c102ab0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.728+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c19efb0 con 0x7fe29c102ab0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.728+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294005740 con 0x7fe29c102ab0 2024-05-08T21:53:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.728+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c19f490 con 0x7fe29c102ab0 2024-05-08T21:53:25.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.729+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c108450 con 0x7fe29c102ab0 2024-05-08T21:53:25.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.730+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe29400dc60 con 0x7fe29c102ab0 2024-05-08T21:53:25.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.730+0000 7fe299ffb700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe28c0a1120 0x7fe28c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.731+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2940cbfc0 con 0x7fe29c102ab0 2024-05-08T21:53:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.731+0000 7fe2a0837700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe28c0a1120 0x7fe28c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.731+0000 7fe2a0837700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe28c0a1120 0x7fe28c0a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe290006010 tx=0x7fe2900037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.735+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe294095800 con 0x7fe29c102ab0 2024-05-08T21:53:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:25.992+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fe29c02cda0 con 0x7fe29c102ab0 2024-05-08T21:53:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.578+0000 7fe299ffb700 1 -- 172.21.15.64:0/514526246 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe294098e20 con 0x7fe29c102ab0 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.581+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe28c0a1120 msgr2=0x7fe28c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.581+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe28c0a1120 0x7fe28c0a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe290006010 tx=0x7fe2900037b0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.581+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 msgr2=0x7fe29c19e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.581+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 0x7fe29c19e490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe294007ee0 tx=0x7fe29400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 shutdown_connections 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe28c0a1120 0x7fe28c0a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe29c102ab0 0x7fe29c19e490 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe29c100f20 0x7fe29c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 --2- 172.21.15.64:0/514526246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe29c1020f0 0x7fe29c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 >> 172.21.15.64:0/514526246 conn(0x7fe29c0fc4e0 msgr2=0x7fe29c105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 shutdown_connections 2024-05-08T21:53:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.582+0000 7fe2a2a9b700 1 -- 172.21.15.64:0/514526246 wait complete. 2024-05-08T21:53:26.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T21:53:26.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 -- 172.21.15.64:0/3207111321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088101a60 msgr2=0x7fe088103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3207111321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088101a60 0x7fe088103e50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe078009990 tx=0x7fe078009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 -- 172.21.15.64:0/3207111321 shutdown_connections 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3207111321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088106cc0 0x7fe0881090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3207111321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088101a60 0x7fe088103e50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3207111321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088104390 0x7fe088106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 -- 172.21.15.64:0/3207111321 >> 172.21.15.64:0/3207111321 conn(0x7fe0880fb3f0 msgr2=0x7fe0880fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 -- 172.21.15.64:0/3207111321 shutdown_connections 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.735+0000 7fe08df8c700 1 -- 172.21.15.64:0/3207111321 wait complete. 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 Processor -- start 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 -- start start 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088101a60 0x7fe088196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 0x7fe088197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088106cc0 0x7fe08819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08819ea50 con 0x7fe088104390 2024-05-08T21:53:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08819eb90 con 0x7fe088101a60 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe08df8c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08819ed20 con 0x7fe088106cc0 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe086ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 0x7fe088197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.736+0000 7fe086ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 0x7fe088197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46626/0 (socket says 172.21.15.64:46626) 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 -- 172.21.15.64:0/3715804657 learned_addr learned my addr 172.21.15.64:0/3715804657 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe0877fe700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088101a60 0x7fe088196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe087fff700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088106cc0 0x7fe08819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 -- 172.21.15.64:0/3715804657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088106cc0 msgr2=0x7fe08819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088106cc0 0x7fe08819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 -- 172.21.15.64:0/3715804657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088101a60 msgr2=0x7fe088196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088101a60 0x7fe088196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 -- 172.21.15.64:0/3715804657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780095f0 con 0x7fe088104390 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe087fff700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088106cc0 0x7fe08819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe0877fe700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088101a60 0x7fe088196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:26.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe086ffd700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 0x7fe088197460 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00d950 tx=0x7fe07c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c009b40 con 0x7fe088104390 2024-05-08T21:53:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07c009ca0 con 0x7fe088104390 2024-05-08T21:53:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08819efa0 con 0x7fe088104390 2024-05-08T21:53:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00f620 con 0x7fe088104390 2024-05-08T21:53:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.737+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08819f480 con 0x7fe088104390 2024-05-08T21:53:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.738+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0880fcf70 con 0x7fe088104390 2024-05-08T21:53:26.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.739+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe07c010460 con 0x7fe088104390 2024-05-08T21:53:26.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.740+0000 7fe084ff9700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe0700a1120 0x7fe0700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:26.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.740+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe07c0ccb10 con 0x7fe088104390 2024-05-08T21:53:26.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.740+0000 7fe0877fe700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe0700a1120 0x7fe0700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:26.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.740+0000 7fe0877fe700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe0700a1120 0x7fe0700a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe078006010 tx=0x7fe0780037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:26.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:26.743+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe07c0963f0 con 0x7fe088104390 2024-05-08T21:53:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:26 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2359929769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T21:53:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:26 smithi186 ceph-mon[46619]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:26 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/514526246' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T21:53:26.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:26 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2359929769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T21:53:26.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:26 smithi064 ceph-mon[42500]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:26.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:26 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/514526246' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T21:53:27.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.000+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe088067460 con 0x7fe088104390 2024-05-08T21:53:27.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:26 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2359929769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T21:53:27.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:26 smithi160 ceph-mon[46962]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:27.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:26 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/514526246' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T21:53:27.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.586+0000 7fe084ff9700 1 -- 172.21.15.64:0/3715804657 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe07c0200c0 con 0x7fe088104390 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe0700a1120 msgr2=0x7fe0700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe0700a1120 0x7fe0700a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe078006010 tx=0x7fe0780037b0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 msgr2=0x7fe088197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 0x7fe088197460 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00d950 tx=0x7fe07c00dd10 comp rx=0 tx=0).stop 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 shutdown_connections 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe0700a1120 0x7fe0700a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe088101a60 0x7fe088196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe088106cc0 0x7fe08819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 --2- 172.21.15.64:0/3715804657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe088104390 0x7fe088197460 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 >> 172.21.15.64:0/3715804657 conn(0x7fe0880fb3f0 msgr2=0x7fe0881023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:27.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 shutdown_connections 2024-05-08T21:53:27.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.588+0000 7fe08df8c700 1 -- 172.21.15.64:0/3715804657 wait complete. 2024-05-08T21:53:27.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T21:53:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 -- 172.21.15.64:0/3840732286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478102ab0 msgr2=0x7f54781093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3840732286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478102ab0 0x7f54781093d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f547000b0a0 tx=0x7f547000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 -- 172.21.15.64:0/3840732286 shutdown_connections 2024-05-08T21:53:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3840732286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478102ab0 0x7f54781093d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3840732286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478100f20 0x7f5478101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3840732286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54781020f0 0x7f5478102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 -- 172.21.15.64:0/3840732286 >> 172.21.15.64:0/3840732286 conn(0x7f54780fc4e0 msgr2=0x7f54780fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 -- 172.21.15.64:0/3840732286 shutdown_connections 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 -- 172.21.15.64:0/3840732286 wait complete. 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 Processor -- start 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 -- start start 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.738+0000 7f547f4bc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 0x7f5478074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54781020f0 0x7f5478073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478102ab0 0x7f54780738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5478074070 con 0x7f5478100f20 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54780741e0 con 0x7f54781020f0 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54781a1020 con 0x7f5478102ab0 2024-05-08T21:53:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 0x7f5478074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 0x7f5478074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46648/0 (socket says 172.21.15.64:46648) 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 -- 172.21.15.64:0/3847229427 learned_addr learned my addr 172.21.15.64:0/3847229427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547da59700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478102ab0 0x7f54780738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547ca57700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54781020f0 0x7f5478073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 -- 172.21.15.64:0/3847229427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478102ab0 msgr2=0x7f54780738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478102ab0 0x7f54780738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 -- 172.21.15.64:0/3847229427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54781020f0 msgr2=0x7f5478073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54781020f0 0x7f5478073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 -- 172.21.15.64:0/3847229427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009d00 con 0x7f5478100f20 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547d258700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 0x7f5478074d00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f547400d980 tx=0x7f547400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474004020 con 0x7f5478100f20 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f547400f690 con 0x7f5478100f20 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54781a12b0 con 0x7f5478100f20 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474003a60 con 0x7f5478100f20 2024-05-08T21:53:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.739+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54781a1810 con 0x7f5478100f20 2024-05-08T21:53:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.740+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54781083c0 con 0x7f5478100f20 2024-05-08T21:53:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.741+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f547400fb00 con 0x7f5478100f20 2024-05-08T21:53:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.742+0000 7f546e7fc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a11a0 0x7f54640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.742+0000 7f547ca57700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a11a0 0x7f54640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.742+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54740cc5d0 con 0x7f5478100f20 2024-05-08T21:53:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.743+0000 7f547ca57700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a11a0 0x7f54640a3660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5468007be0 tx=0x7f54680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:27.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:27.746+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5474095f30 con 0x7f5478100f20 2024-05-08T21:53:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: pgmap v222: 97 pgs: 15 creating+peering, 59 active+clean, 23 unknown; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T21:53:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T21:53:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/514526246' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T21:53:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T21:53:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:27.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T21:53:27.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T21:53:27.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3715804657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: pgmap v222: 97 pgs: 15 creating+peering, 59 active+clean, 23 unknown; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/514526246' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T21:53:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3715804657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T21:53:28.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.010+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f547810c9d0 con 0x7f5478100f20 2024-05-08T21:53:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: pgmap v222: 97 pgs: 15 creating+peering, 59 active+clean, 23 unknown; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/514526246' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3715804657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T21:53:28.544 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: 2024-05-08T21:53:28.062+0000 7f965138e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T21:53:28.544 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: 2024-05-08T21:53:28.062+0000 7f965138e700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T21:53:28.545 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: 2024-05-08T21:53:28.062+0000 7f965138e700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T21:53:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] up:active 2024-05-08T21:53:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3715804657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T21:53:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T21:53:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] up:active 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-05-08T21:53:28.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3847229427' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T21:53:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.949+0000 7f546e7fc700 1 -- 172.21.15.64:0/3847229427 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5474099550 con 0x7f5478100f20 2024-05-08T21:53:28.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.952+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a11a0 msgr2=0x7f54640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:28.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.952+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a11a0 0x7f54640a3660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5468007be0 tx=0x7f54680073d0 comp rx=0 tx=0).stop 2024-05-08T21:53:28.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.952+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 msgr2=0x7f5478074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:28.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.952+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 0x7f5478074d00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f547400d980 tx=0x7f547400dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:28.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 shutdown_connections 2024-05-08T21:53:28.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a11a0 0x7f54640a3660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:28.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478102ab0 0x7f54780738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100f20 0x7f5478074d00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 --2- 172.21.15.64:0/3847229427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54781020f0 0x7f5478073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 >> 172.21.15.64:0/3847229427 conn(0x7f54780fc4e0 msgr2=0x7f5478105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 shutdown_connections 2024-05-08T21:53:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:28.953+0000 7f547f4bc700 1 -- 172.21.15.64:0/3847229427 wait complete. 2024-05-08T21:53:28.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] up:active 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3715804657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] up:active 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-05-08T21:53:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3847229427' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T21:53:28.972 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-08T21:53:28.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T21:53:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] up:active 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3715804657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] up:active 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-05-08T21:53:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3847229427' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T21:53:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.107+0000 7f5024507700 1 -- 172.21.15.64:0/542637155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c102ab0 msgr2=0x7f501c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.107+0000 7f5024507700 1 --2- 172.21.15.64:0/542637155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c102ab0 0x7f501c1093d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f501400b0a0 tx=0x7f501400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- 172.21.15.64:0/542637155 shutdown_connections 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 --2- 172.21.15.64:0/542637155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f501c100f20 0x7f501c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 --2- 172.21.15.64:0/542637155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c102ab0 0x7f501c1093d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 --2- 172.21.15.64:0/542637155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 0x7f501c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- 172.21.15.64:0/542637155 >> 172.21.15.64:0/542637155 conn(0x7f501c0fc4e0 msgr2=0x7f501c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- 172.21.15.64:0/542637155 shutdown_connections 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- 172.21.15.64:0/542637155 wait complete. 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 Processor -- start 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- start start 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 0x7f501c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 0x7f501c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f501c102ab0 0x7f501c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19eac0 con 0x7f501c100f20 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19ec00 con 0x7f501c102ab0 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f5024507700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19ed90 con 0x7f501c1020f0 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.108+0000 7f50222a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 0x7f501c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 0x7f501c196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46662/0 (socket says 172.21.15.64:46662) 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 -- 172.21.15.64:0/3255374765 learned_addr learned my addr 172.21.15.64:0/3255374765 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f5022aa4700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f501c102ab0 0x7f501c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f5021aa2700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 0x7f501c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 -- 172.21.15.64:0/3255374765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 msgr2=0x7f501c1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 0x7f501c1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 -- 172.21.15.64:0/3255374765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f501c102ab0 msgr2=0x7f501c19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f501c102ab0 0x7f501c19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 -- 172.21.15.64:0/3255374765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014009d00 con 0x7f501c100f20 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f5021aa2700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 0x7f501c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50222a3700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 0x7f501c196f90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f501800b500 tx=0x7f501800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018011830 con 0x7f501c100f20 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c19f070 con 0x7f501c100f20 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5018011e50 con 0x7f501c100f20 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800f2e0 con 0x7f501c100f20 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.109+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c19f5d0 con 0x7f501c100f20 2024-05-08T21:53:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.110+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c1082e0 con 0x7f501c100f20 2024-05-08T21:53:29.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.111+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5018010580 con 0x7f501c100f20 2024-05-08T21:53:29.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.112+0000 7f50137fe700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f50080a1120 0x7f50080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:29.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.112+0000 7f5021aa2700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f50080a1120 0x7f50080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:29.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.112+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50180cc820 con 0x7f501c100f20 2024-05-08T21:53:29.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.113+0000 7f5021aa2700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f50080a1120 0x7f50080a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f500c005fd0 tx=0x7f500c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.116+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5018096060 con 0x7f501c100f20 2024-05-08T21:53:29.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.359+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f501c0613e0 con 0x7f50080a1120 2024-05-08T21:53:29.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:53:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:53:29.363+0000 7f0d76460700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:53:29.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:53:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:53:29.363+0000 7f0d76460700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:53:29.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:53:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:53:29.363+0000 7f0d76460700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:53:29.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:53:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:53:29.363+0000 7f0d76460700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:53:29.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 21:53:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T21:53:29.363+0000 7f0d76460700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:53:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.977+0000 7f50137fe700 1 -- 172.21.15.64:0/3255374765 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f501c0613e0 con 0x7f50080a1120 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f50080a1120 msgr2=0x7f50080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f50080a1120 0x7f50080a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f500c005fd0 tx=0x7f500c005e30 comp rx=0 tx=0).stop 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 msgr2=0x7f501c196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 0x7f501c196f90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f501800b500 tx=0x7f501800b810 comp rx=0 tx=0).stop 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 shutdown_connections 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f50080a1120 0x7f50080a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f501c102ab0 0x7f501c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f501c100f20 0x7f501c196f90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 --2- 172.21.15.64:0/3255374765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f501c1020f0 0x7f501c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 >> 172.21.15.64:0/3255374765 conn(0x7f501c0fc4e0 msgr2=0x7f501c105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 shutdown_connections 2024-05-08T21:53:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:29.980+0000 7f5024507700 1 -- 172.21.15.64:0/3255374765 wait complete. 2024-05-08T21:53:29.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 -- 172.21.15.64:0/3290163546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90800ffe60 msgr2=0x7f9080100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 --2- 172.21.15.64:0/3290163546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90800ffe60 0x7f9080100280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9070009990 tx=0x7f9070009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 -- 172.21.15.64:0/3290163546 shutdown_connections 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 --2- 172.21.15.64:0/3290163546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9080101970 0x7f9080108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 --2- 172.21.15.64:0/3290163546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90800ffe60 0x7f9080100280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 --2- 172.21.15.64:0/3290163546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080100fb0 0x7f9080101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 -- 172.21.15.64:0/3290163546 >> 172.21.15.64:0/3290163546 conn(0x7f90800fb3f0 msgr2=0x7f90800fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 -- 172.21.15.64:0/3290163546 shutdown_connections 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.134+0000 7f9085834700 1 -- 172.21.15.64:0/3290163546 wait complete. 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 Processor -- start 2024-05-08T21:53:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 -- start start 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90800ffe60 0x7f9080196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 0x7f9080197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080101970 0x7f908019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908019e9f0 con 0x7f9080100fb0 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908019eb30 con 0x7f9080101970 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f9085834700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908019ecc0 con 0x7f90800ffe60 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f907e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 0x7f9080197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f907e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 0x7f9080197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46692/0 (socket says 172.21.15.64:46692) 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.135+0000 7f907e7fc700 1 -- 172.21.15.64:0/1537708028 learned_addr learned my addr 172.21.15.64:0/1537708028 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907effd700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90800ffe60 0x7f9080196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907f7fe700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080101970 0x7f908019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907e7fc700 1 -- 172.21.15.64:0/1537708028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90800ffe60 msgr2=0x7f9080196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907e7fc700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90800ffe60 0x7f9080196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907e7fc700 1 -- 172.21.15.64:0/1537708028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080101970 msgr2=0x7f908019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907e7fc700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080101970 0x7f908019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907e7fc700 1 -- 172.21.15.64:0/1537708028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700095f0 con 0x7f9080100fb0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f907e7fc700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 0x7f9080197400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f907400d980 tx=0x7f907400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074004220 con 0x7f9080100fb0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9074007640 con 0x7f9080100fb0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908019efa0 con 0x7f9080100fb0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074005690 con 0x7f9080100fb0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.136+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908019f470 con 0x7f9080100fb0 2024-05-08T21:53:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.137+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90801072d0 con 0x7f9080100fb0 2024-05-08T21:53:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.138+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9074004750 con 0x7f9080100fb0 2024-05-08T21:53:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.139+0000 7f9084832700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f90680a11a0 0x7f90680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.139+0000 7f907effd700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f90680a11a0 0x7f90680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.139+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f90740cc5b0 con 0x7f9080100fb0 2024-05-08T21:53:30.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.139+0000 7f907effd700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f90680a11a0 0x7f90680a3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f907000b5c0 tx=0x7f9070003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.143+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9074095df0 con 0x7f9080100fb0 2024-05-08T21:53:30.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: pgmap v223: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] up:active 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3847229427' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: Cluster is now healthy 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-05-08T21:53:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:29 smithi064 ceph-mon[42500]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-05-08T21:53:30.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: pgmap v223: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-08T21:53:30.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] up:active 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3847229427' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: Cluster is now healthy 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-05-08T21:53:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:29 smithi160 ceph-mon[46962]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-05-08T21:53:30.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.390+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f908002ce10 con 0x7f90680a11a0 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: pgmap v223: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] up:active 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3847229427' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: Cluster is now healthy 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-05-08T21:53:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:29 smithi186 ceph-mon[46619]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-05-08T21:53:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.413+0000 7f9084832700 1 -- 172.21.15.64:0/1537708028 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f908002ce10 con 0x7f90680a11a0 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f90680a11a0 msgr2=0x7f90680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f90680a11a0 0x7f90680a3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f907000b5c0 tx=0x7f9070003790 comp rx=0 tx=0).stop 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 msgr2=0x7f9080197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 0x7f9080197400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f907400d980 tx=0x7f907400dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 shutdown_connections 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f90680a11a0 0x7f90680a3660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080101970 0x7f908019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90800ffe60 0x7f9080196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 --2- 172.21.15.64:0/1537708028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080100fb0 0x7f9080197400 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 >> 172.21.15.64:0/1537708028 conn(0x7f90800fb3f0 msgr2=0x7f9080104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 shutdown_connections 2024-05-08T21:53:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.415+0000 7f9085834700 1 -- 172.21.15.64:0/1537708028 wait complete. 2024-05-08T21:53:30.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 -- 172.21.15.64:0/2874744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0100fd0 msgr2=0x7f30f0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 --2- 172.21.15.64:0/2874744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0100fd0 0x7f30f0101450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009990 tx=0x7f30e0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 -- 172.21.15.64:0/2874744578 shutdown_connections 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 --2- 172.21.15.64:0/2874744578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0101990 0x7f30f01082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 --2- 172.21.15.64:0/2874744578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 0x7f30f01002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 --2- 172.21.15.64:0/2874744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0100fd0 0x7f30f0101450 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.558+0000 7f30f771c700 1 -- 172.21.15.64:0/2874744578 >> 172.21.15.64:0/2874744578 conn(0x7f30f00fb3f0 msgr2=0x7f30f00fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 -- 172.21.15.64:0/2874744578 shutdown_connections 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 -- 172.21.15.64:0/2874744578 wait complete. 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 Processor -- start 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 -- start start 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 0x7f30f0069d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0100fd0 0x7f30f019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 0x7f30f019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f019e980 con 0x7f30f0101990 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f019eaf0 con 0x7f30f0100fd0 2024-05-08T21:53:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f771c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f019ecb0 con 0x7f30f00ffe80 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f5cb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 0x7f30f019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f5cb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 0x7f30f019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46702/0 (socket says 172.21.15.64:46702) 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.559+0000 7f30f5cb9700 1 -- 172.21.15.64:0/1094444255 learned_addr learned my addr 172.21.15.64:0/1094444255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f4cb7700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0100fd0 0x7f30f019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f54b8700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 0x7f30f0069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f5cb9700 1 -- 172.21.15.64:0/1094444255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 msgr2=0x7f30f0069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f5cb9700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 0x7f30f0069d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f5cb9700 1 -- 172.21.15.64:0/1094444255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0100fd0 msgr2=0x7f30f019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f5cb9700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0100fd0 0x7f30f019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f5cb9700 1 -- 172.21.15.64:0/1094444255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00095f0 con 0x7f30f0101990 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f4cb7700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0100fd0 0x7f30f019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f54b8700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 0x7f30f0069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f5cb9700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 0x7f30f019e190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30e8007ee0 tx=0x7f30e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:30.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8017070 con 0x7f30f0101990 2024-05-08T21:53:30.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30e800f040 con 0x7f30f0101990 2024-05-08T21:53:30.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f019ef30 con 0x7f30f0101990 2024-05-08T21:53:30.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8005740 con 0x7f30f0101990 2024-05-08T21:53:30.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.560+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f019f360 con 0x7f30f0101990 2024-05-08T21:53:30.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.561+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f0107240 con 0x7f30f0101990 2024-05-08T21:53:30.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.562+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f30e80058a0 con 0x7f30f0101990 2024-05-08T21:53:30.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.563+0000 7f30e67fc700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30dc0a1120 0x7f30dc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.563+0000 7f30f54b8700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30dc0a1120 0x7f30dc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.563+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30e80cc650 con 0x7f30f0101990 2024-05-08T21:53:30.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.564+0000 7f30f54b8700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30dc0a1120 0x7f30dc0a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005fd0 tx=0x7f30ec005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:30.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.567+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f30e8095f30 con 0x7f30f0101990 2024-05-08T21:53:30.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.814+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f30f002ce10 con 0x7f30dc0a1120 2024-05-08T21:53:30.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.818+0000 7f30e67fc700 1 -- 172.21.15.64:0/1094444255 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f30f002ce10 con 0x7f30dc0a1120 2024-05-08T21:53:30.820 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_0/db95522f-3ddf-4784-9eb2-d89e8fda1b3c 2024-05-08T21:53:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30dc0a1120 msgr2=0x7f30dc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30dc0a1120 0x7f30dc0a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005fd0 tx=0x7f30ec005e20 comp rx=0 tx=0).stop 2024-05-08T21:53:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 msgr2=0x7f30f019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 0x7f30f019e190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30e8007ee0 tx=0x7f30e800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 shutdown_connections 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30dc0a1120 0x7f30dc0a35e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30f0101990 0x7f30f019e190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30f00ffe80 0x7f30f0069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 --2- 172.21.15.64:0/1094444255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30f0100fd0 0x7f30f019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 >> 172.21.15.64:0/1094444255 conn(0x7f30f00fb3f0 msgr2=0x7f30f0104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 shutdown_connections 2024-05-08T21:53:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.820+0000 7f30f771c700 1 -- 172.21.15.64:0/1094444255 wait complete. 2024-05-08T21:53:30.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T21:53:30.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 -- 172.21.15.64:0/2903189790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 msgr2=0x7f56841013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 --2- 172.21.15.64:0/2903189790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f56841013a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56800099c0 tx=0x7f5680009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 -- 172.21.15.64:0/2903189790 shutdown_connections 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 --2- 172.21.15.64:0/2903189790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5684102b10 0x7f56841093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 --2- 172.21.15.64:0/2903189790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f56841013a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 --2- 172.21.15.64:0/2903189790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5684102170 0x7f56841025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 -- 172.21.15.64:0/2903189790 >> 172.21.15.64:0/2903189790 conn(0x7f56840fc4f0 msgr2=0x7f56840fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 -- 172.21.15.64:0/2903189790 shutdown_connections 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 -- 172.21.15.64:0/2903189790 wait complete. 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 Processor -- start 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.966+0000 7f568c707700 1 -- start start 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f5684196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5684102170 0x7f5684197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5684102b10 0x7f568419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568419ea40 con 0x7f5684100f80 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568419eb80 con 0x7f5684102b10 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568419ed10 con 0x7f5684102170 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f5684196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f5684196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46730/0 (socket says 172.21.15.64:46730) 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 -- 172.21.15.64:0/236712043 learned_addr learned my addr 172.21.15.64:0/236712043 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f5689ca2700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5684102170 0x7f5684197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568aca4700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5684102b10 0x7f568419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 -- 172.21.15.64:0/236712043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5684102170 msgr2=0x7f5684197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5684102170 0x7f5684197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 -- 172.21.15.64:0/236712043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5684102b10 msgr2=0x7f568419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5684102b10 0x7f568419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:30.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 -- 172.21.15.64:0/236712043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800095f0 con 0x7f5684100f80 2024-05-08T21:53:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568a4a3700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f5684196f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5680003820 tx=0x7f5680004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568001d070 con 0x7f5684100f80 2024-05-08T21:53:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5680004ca0 con 0x7f5684100f80 2024-05-08T21:53:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568419ef90 con 0x7f5684100f80 2024-05-08T21:53:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.967+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680003d30 con 0x7f5684100f80 2024-05-08T21:53:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.968+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568419f350 con 0x7f5684100f80 2024-05-08T21:53:30.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.968+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684108450 con 0x7f5684100f80 2024-05-08T21:53:30.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.970+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5680005030 con 0x7f5684100f80 2024-05-08T21:53:30.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.970+0000 7f567b7fe700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56700a1120 0x7f56700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:30.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.970+0000 7f5689ca2700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56700a1120 0x7f56700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:30.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.970+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56800cd880 con 0x7f5684100f80 2024-05-08T21:53:30.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.971+0000 7f5689ca2700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56700a1120 0x7f56700a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5674005950 tx=0x7f567400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:30.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:30.974+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56800971f0 con 0x7f5684100f80 2024-05-08T21:53:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:30 smithi064 ceph-mon[42500]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:30 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] up:active 2024-05-08T21:53:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:30 smithi064 ceph-mon[42500]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-05-08T21:53:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:30 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:53:31.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.220+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f568402ce10 con 0x7f56700a1120 2024-05-08T21:53:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.245+0000 7f567b7fe700 1 -- 172.21.15.64:0/236712043 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f568402ce10 con 0x7f56700a1120 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56700a1120 msgr2=0x7f56700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56700a1120 0x7f56700a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5674005950 tx=0x7f567400b410 comp rx=0 tx=0).stop 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 msgr2=0x7f5684196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f5684196f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5680003820 tx=0x7f5680004730 comp rx=0 tx=0).stop 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 shutdown_connections 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56700a1120 0x7f56700a35e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5684102b10 0x7f568419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5684100f80 0x7f5684196f10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 --2- 172.21.15.64:0/236712043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5684102170 0x7f5684197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 >> 172.21.15.64:0/236712043 conn(0x7f56840fc4f0 msgr2=0x7f5684105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 shutdown_connections 2024-05-08T21:53:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.247+0000 7f568c707700 1 -- 172.21.15.64:0/236712043 wait complete. 2024-05-08T21:53:31.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T21:53:31.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:30 smithi160 ceph-mon[46962]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:31.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:30 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] up:active 2024-05-08T21:53:31.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:30 smithi160 ceph-mon[46962]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-05-08T21:53:31.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:30 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:53:31.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:30 smithi186 ceph-mon[46619]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:31.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:30 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] up:active 2024-05-08T21:53:31.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:30 smithi186 ceph-mon[46619]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-05-08T21:53:31.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:30 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:53:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 -- 172.21.15.64:0/2645705516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef0100fb0 msgr2=0x7f4ef0101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 --2- 172.21.15.64:0/2645705516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0101430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009990 tx=0x7f4ee0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 -- 172.21.15.64:0/2645705516 shutdown_connections 2024-05-08T21:53:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 --2- 172.21.15.64:0/2645705516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 --2- 172.21.15.64:0/2645705516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0101970 0x7f4ef0108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 --2- 172.21.15.64:0/2645705516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0101430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 -- 172.21.15.64:0/2645705516 >> 172.21.15.64:0/2645705516 conn(0x7f4ef00fb3f0 msgr2=0x7f4ef00fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 -- 172.21.15.64:0/2645705516 shutdown_connections 2024-05-08T21:53:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.391+0000 7f4ef4869700 1 -- 172.21.15.64:0/2645705516 wait complete. 2024-05-08T21:53:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 Processor -- start 2024-05-08T21:53:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 -- start start 2024-05-08T21:53:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0103d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0101970 0x7f4ef0102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0103060 con 0x7f4ef00ffe60 2024-05-08T21:53:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01031d0 con 0x7f4ef0100fb0 2024-05-08T21:53:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4ef4869700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01a0ee0 con 0x7f4ef0101970 2024-05-08T21:53:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4eee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0103d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4eee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0103d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46758/0 (socket says 172.21.15.64:46758) 2024-05-08T21:53:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.392+0000 7f4eee59c700 1 -- 172.21.15.64:0/3304329008 learned_addr learned my addr 172.21.15.64:0/3304329008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eedd9b700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eeed9d700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0101970 0x7f4ef0102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eee59c700 1 -- 172.21.15.64:0/3304329008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0101970 msgr2=0x7f4ef0102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eee59c700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0101970 0x7f4ef0102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eee59c700 1 -- 172.21.15.64:0/3304329008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0100fb0 msgr2=0x7f4ef0102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eee59c700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eee59c700 1 -- 172.21.15.64:0/3304329008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00095f0 con 0x7f4ef00ffe60 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eedd9b700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4eee59c700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0103d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c9b0 tx=0x7f4ed800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8007b00 con 0x7f4ef00ffe60 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed8007c60 con 0x7f4ef00ffe60 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef01a1140 con 0x7f4ef00ffe60 2024-05-08T21:53:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8018660 con 0x7f4ef00ffe60 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.393+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01a16a0 con 0x7f4ef00ffe60 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.394+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0067cf0 con 0x7f4ef00ffe60 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.395+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4ed8018880 con 0x7f4ef00ffe60 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.396+0000 7f4ee77fe700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4edc0a11a0 0x7f4edc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.396+0000 7f4eedd9b700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4edc0a11a0 0x7f4edc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.396+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4ed80cc790 con 0x7f4ef00ffe60 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.396+0000 7f4eedd9b700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4edc0a11a0 0x7f4edc0a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000b5c0 tx=0x7f4ee0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.400+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4ed8096070 con 0x7f4ef00ffe60 2024-05-08T21:53:31.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.652+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ef00fd5c0 con 0x7f4edc0a11a0 2024-05-08T21:53:31.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.655+0000 7f4ee77fe700 1 -- 172.21.15.64:0/3304329008 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f4ef00fd5c0 con 0x7f4edc0a11a0 2024-05-08T21:53:31.657 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_1/2e3115a7-b916-4b6b-84de-afba6db85011 2024-05-08T21:53:31.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4edc0a11a0 msgr2=0x7f4edc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4edc0a11a0 0x7f4edc0a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000b5c0 tx=0x7f4ee0005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 msgr2=0x7f4ef0103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0103d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c9b0 tx=0x7f4ed800cd70 comp rx=0 tx=0).stop 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 shutdown_connections 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4edc0a11a0 0x7f4edc0a3660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0101970 0x7f4ef0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ef00ffe60 0x7f4ef0103d40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 --2- 172.21.15.64:0/3304329008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0100fb0 0x7f4ef0102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 >> 172.21.15.64:0/3304329008 conn(0x7f4ef00fb3f0 msgr2=0x7f4ef00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 shutdown_connections 2024-05-08T21:53:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.658+0000 7f4ef4869700 1 -- 172.21.15.64:0/3304329008 wait complete. 2024-05-08T21:53:31.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:53:31.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 -- 172.21.15.64:0/1507053896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe300ffd60 msgr2=0x7efe30100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 --2- 172.21.15.64:0/1507053896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe300ffd60 0x7efe30100180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe200099c0 tx=0x7efe20009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 -- 172.21.15.64:0/1507053896 shutdown_connections 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 --2- 172.21.15.64:0/1507053896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe301018f0 0x7efe301081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 --2- 172.21.15.64:0/1507053896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe300ffd60 0x7efe30100180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 --2- 172.21.15.64:0/1507053896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 0x7efe301013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 -- 172.21.15.64:0/1507053896 >> 172.21.15.64:0/1507053896 conn(0x7efe300fb2f0 msgr2=0x7efe300fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 -- 172.21.15.64:0/1507053896 shutdown_connections 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.796+0000 7efe3620a700 1 -- 172.21.15.64:0/1507053896 wait complete. 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 Processor -- start 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 -- start start 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe300ffd60 0x7efe30196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 0x7efe301973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 0x7efe3019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3019e9d0 con 0x7efe301018f0 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3019eb10 con 0x7efe300ffd60 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe3620a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3019eca0 con 0x7efe30100f50 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe2ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 0x7efe3019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe2ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 0x7efe3019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46770/0 (socket says 172.21.15.64:46770) 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe2ffff700 1 -- 172.21.15.64:0/779800366 learned_addr learned my addr 172.21.15.64:0/779800366 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.797+0000 7efe2f7fe700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe300ffd60 0x7efe30196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe26ffd700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 0x7efe301973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2ffff700 1 -- 172.21.15.64:0/779800366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 msgr2=0x7efe301973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2ffff700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 0x7efe301973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2ffff700 1 -- 172.21.15.64:0/779800366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe300ffd60 msgr2=0x7efe30196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2ffff700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe300ffd60 0x7efe30196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2ffff700 1 -- 172.21.15.64:0/779800366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200095f0 con 0x7efe301018f0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe26ffd700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 0x7efe301973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2ffff700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 0x7efe3019e400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efe28007ee0 tx=0x7efe2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28017070 con 0x7efe301018f0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe2800f040 con 0x7efe301018f0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3019ef80 con 0x7efe301018f0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28005740 con 0x7efe301018f0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.798+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3019f450 con 0x7efe301018f0 2024-05-08T21:53:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.799+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30067c10 con 0x7efe301018f0 2024-05-08T21:53:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.800+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efe2800dc60 con 0x7efe301018f0 2024-05-08T21:53:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.801+0000 7efe2d7fa700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe1c0a11a0 0x7efe1c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.801+0000 7efe2f7fe700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe1c0a11a0 0x7efe1c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.801+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efe280cc510 con 0x7efe301018f0 2024-05-08T21:53:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.802+0000 7efe2f7fe700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe1c0a11a0 0x7efe1c0a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe20009f50 tx=0x7efe20005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.805+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efe28095df0 con 0x7efe301018f0 2024-05-08T21:53:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:31.982+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efe280961c0 con 0x7efe301018f0 2024-05-08T21:53:32.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.068+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe3010b890 con 0x7efe301018f0 2024-05-08T21:53:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.068+0000 7efe2d7fa700 1 -- 172.21.15.64:0/779800366 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10921 (secure 0 0 0) 0x7efe28099880 con 0x7efe301018f0 2024-05-08T21:53:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:32.070 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T21:53:29.959964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe1c0a11a0 msgr2=0x7efe1c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe1c0a11a0 0x7efe1c0a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe20009f50 tx=0x7efe20005bc0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 msgr2=0x7efe3019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 0x7efe3019e400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efe28007ee0 tx=0x7efe2800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 shutdown_connections 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe1c0a11a0 0x7efe1c0a3660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe300ffd60 0x7efe30196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe301018f0 0x7efe3019e400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 --2- 172.21.15.64:0/779800366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30100f50 0x7efe301973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 >> 172.21.15.64:0/779800366 conn(0x7efe300fb2f0 msgr2=0x7efe30067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 shutdown_connections 2024-05-08T21:53:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.071+0000 7efe3620a700 1 -- 172.21.15.64:0/779800366 wait complete. 2024-05-08T21:53:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:53:32.085 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 -- 172.21.15.64:0/1935347563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 msgr2=0x7f03d8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 --2- 172.21.15.64:0/1935347563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8102440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009990 tx=0x7f03d4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 -- 172.21.15.64:0/1935347563 shutdown_connections 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 --2- 172.21.15.64:0/1935347563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d8100e70 0x7f03d8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 --2- 172.21.15.64:0/1935347563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03d8102980 0x7f03d81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 --2- 172.21.15.64:0/1935347563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8102440 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 -- 172.21.15.64:0/1935347563 >> 172.21.15.64:0/1935347563 conn(0x7f03d80fc400 msgr2=0x7f03d80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 -- 172.21.15.64:0/1935347563 shutdown_connections 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 -- 172.21.15.64:0/1935347563 wait complete. 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 Processor -- start 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.211+0000 7f03e034b700 1 -- start start 2024-05-08T21:53:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03e034b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03d8100e70 0x7f03d8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03e034b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03e034b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d8102980 0x7f03d819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03e034b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d819e8f0 con 0x7f03d8101fc0 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03e034b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d819ea60 con 0x7f03d8102980 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03e034b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d819ec20 con 0x7f03d8100e70 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46796/0 (socket says 172.21.15.64:46796) 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 -- 172.21.15.64:0/2837927255 learned_addr learned my addr 172.21.15.64:0/2837927255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03de8e8700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d8102980 0x7f03d819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03de0e7700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03d8100e70 0x7f03d8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 -- 172.21.15.64:0/2837927255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03d8100e70 msgr2=0x7f03d8196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03d8100e70 0x7f03d8196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 -- 172.21.15.64:0/2837927255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d8102980 msgr2=0x7f03d819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d8102980 0x7f03d819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 -- 172.21.15.64:0/2837927255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40095f0 con 0x7f03d8101fc0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03dd8e6700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8197340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03d400b5c0 tx=0x7f03d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.212+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d401d070 con 0x7f03d8101fc0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.213+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d819eea0 con 0x7f03d8101fc0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.213+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03d4003f60 con 0x7f03d8101fc0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.213+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4022d10 con 0x7f03d8101fc0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.213+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d80731d0 con 0x7f03d8101fc0 2024-05-08T21:53:32.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.213+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d81082b0 con 0x7f03d8101fc0 2024-05-08T21:53:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.215+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f03d40040d0 con 0x7f03d8101fc0 2024-05-08T21:53:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.215+0000 7f03cf7fe700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f03c40a55c0 0x7f03c40a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.215+0000 7f03de0e7700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f03c40a55c0 0x7f03c40a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.215+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03d40cdc90 con 0x7f03d8101fc0 2024-05-08T21:53:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.216+0000 7f03de0e7700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f03c40a55c0 0x7f03c40a7a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005950 tx=0x7f03c80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.219+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f03d405dd60 con 0x7f03d8101fc0 2024-05-08T21:53:32.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:31 smithi160 ceph-mon[46962]: pgmap v224: 97 pgs: 97 active+clean; 583 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.7 KiB/s wr, 18 op/s 2024-05-08T21:53:32.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:31 smithi160 ceph-mon[46962]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:32.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:31 smithi160 ceph-mon[46962]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:32.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:31 smithi186 ceph-mon[46619]: pgmap v224: 97 pgs: 97 active+clean; 583 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.7 KiB/s wr, 18 op/s 2024-05-08T21:53:32.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:31 smithi186 ceph-mon[46619]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:32.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:31 smithi186 ceph-mon[46619]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:32.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.429+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03d8067380 con 0x7f03d8101fc0 2024-05-08T21:53:32.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.430+0000 7f03cf7fe700 1 -- 172.21.15.64:0/2837927255 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f03d409af90 con 0x7f03d8101fc0 2024-05-08T21:53:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:32.432 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:24.557465+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f03c40a55c0 msgr2=0x7f03c40a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f03c40a55c0 0x7f03c40a7a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005950 tx=0x7f03c80058e0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 msgr2=0x7f03d8197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8197340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03d400b5c0 tx=0x7f03d4004730 comp rx=0 tx=0).stop 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 shutdown_connections 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f03c40a55c0 0x7f03c40a7a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d8102980 0x7f03d819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03d8100e70 0x7f03d8196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 --2- 172.21.15.64:0/2837927255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d8101fc0 0x7f03d8197340 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 >> 172.21.15.64:0/2837927255 conn(0x7f03d80fc400 msgr2=0x7f03d81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 shutdown_connections 2024-05-08T21:53:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.432+0000 7f03e034b700 1 -- 172.21.15.64:0/2837927255 wait complete. 2024-05-08T21:53:32.445 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:31 smithi064 ceph-mon[42500]: pgmap v224: 97 pgs: 97 active+clean; 583 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.7 KiB/s wr, 18 op/s 2024-05-08T21:53:32.445 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:31 smithi064 ceph-mon[42500]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:32.445 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:31 smithi064 ceph-mon[42500]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:32.447 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T21:53:32.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:53:32.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.571+0000 7f97306a9700 1 -- 172.21.15.64:0/2395573710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97280ffd70 msgr2=0x7f9728100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.571+0000 7f97306a9700 1 --2- 172.21.15.64:0/2395573710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97280ffd70 0x7f9728100190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f97240099c0 tx=0x7f9724009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- 172.21.15.64:0/2395573710 shutdown_connections 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 --2- 172.21.15.64:0/2395573710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728101900 0x7f97281081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 --2- 172.21.15.64:0/2395573710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97280ffd70 0x7f9728100190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 --2- 172.21.15.64:0/2395573710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728100f60 0x7f97281013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- 172.21.15.64:0/2395573710 >> 172.21.15.64:0/2395573710 conn(0x7f97280fb2e0 msgr2=0x7f97280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- 172.21.15.64:0/2395573710 shutdown_connections 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- 172.21.15.64:0/2395573710 wait complete. 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 Processor -- start 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- start start 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97280ffd70 0x7f9728196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 0x7f9728197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728101900 0x7f972819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819e970 con 0x7f9728100f60 2024-05-08T21:53:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819eab0 con 0x7f9728101900 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f97306a9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819ec40 con 0x7f97280ffd70 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.572+0000 7f972dc44700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 0x7f9728197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 0x7f9728197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59742/0 (socket says 172.21.15.64:59742) 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 -- 172.21.15.64:0/3644846339 learned_addr learned my addr 172.21.15.64:0/3644846339 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972e445700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97280ffd70 0x7f9728196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972ec46700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728101900 0x7f972819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 -- 172.21.15.64:0/3644846339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97280ffd70 msgr2=0x7f9728196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97280ffd70 0x7f9728196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 -- 172.21.15.64:0/3644846339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728101900 msgr2=0x7f972819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728101900 0x7f972819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 -- 172.21.15.64:0/3644846339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240095f0 con 0x7f9728100f60 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972ec46700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728101900 0x7f972819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f972dc44700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 0x7f9728197380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f971800e9a0 tx=0x7f971800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718009790 con 0x7f9728100f60 2024-05-08T21:53:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971800cbd0 con 0x7f9728100f60 2024-05-08T21:53:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972819ef20 con 0x7f9728100f60 2024-05-08T21:53:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718018680 con 0x7f9728100f60 2024-05-08T21:53:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.573+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972819f3f0 con 0x7f9728100f60 2024-05-08T21:53:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.574+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9728067380 con 0x7f9728100f60 2024-05-08T21:53:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.575+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f971800cd40 con 0x7f9728100f60 2024-05-08T21:53:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.576+0000 7f971f7fe700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97140a1230 0x7f97140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.576+0000 7f972e445700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97140a1230 0x7f97140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.576+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9718014070 con 0x7f9728100f60 2024-05-08T21:53:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.576+0000 7f972e445700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97140a1230 0x7f97140a36f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9724003730 tx=0x7f9724000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.580+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9718052d60 con 0x7f9728100f60 2024-05-08T21:53:32.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.784+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97280699d0 con 0x7f97140a1230 2024-05-08T21:53:32.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.795+0000 7f971f7fe700 1 -- 172.21.15.64:0/3644846339 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332512 (secure 0 0 0) 0x7f97280699d0 con 0x7f97140a1230 2024-05-08T21:53:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97140a1230 msgr2=0x7f97140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97140a1230 0x7f97140a36f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9724003730 tx=0x7f9724000bc0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 msgr2=0x7f9728197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 0x7f9728197380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f971800e9a0 tx=0x7f971800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 shutdown_connections 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97140a1230 0x7f97140a36f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9728101900 0x7f972819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97280ffd70 0x7f9728196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 --2- 172.21.15.64:0/3644846339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9728100f60 0x7f9728197380 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 >> 172.21.15.64:0/3644846339 conn(0x7f97280fb2e0 msgr2=0x7f97280692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:32.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 shutdown_connections 2024-05-08T21:53:32.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.799+0000 7f97306a9700 1 -- 172.21.15.64:0/3644846339 wait complete. 2024-05-08T21:53:32.808 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-08T21:53:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-05-08T21:53:32.095930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604339,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":162,"num_read_kb":138,"num_write":342,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356672,"kb_used_data":11976,"kb_used_omap":18,"kb_used_meta":344557,"kb_avail":1124585152,"statfs":{"total":1151940427776,"available":1151575195648,"internally_reserved":0,"allocated":12263424,"data_stored":6713547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":352826693},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13971,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":36,"num_read_kb":29,"num_write":161,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.538118"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703161+0000","last_change":"2024-05-08T21:53:23.579256+0000","last_active":"2024-05-08T21:53:23.703161+0000","last_peered":"2024-05-08T21:53:23.703161+0000","last_clean":"2024-05-08T21:53:23.703161+0000","last_became_active":"2024-05-08T21:53:23.578867+0000","last_became_peered":"2024-05-08T21:53:23.578867+0000","last_unstale":"2024-05-08T21:53:23.703161+0000","last_undegraded":"2024-05-08T21:53:23.703161+0000","last_fullsized":"2024-05-08T21:53:23.703161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:07.123241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780009+0000","last_change":"2024-05-08T21:53:23.575229+0000","last_active":"2024-05-08T21:53:23.780009+0000","last_peered":"2024-05-08T21:53:23.780009+0000","last_clean":"2024-05-08T21:53:23.780009+0000","last_became_active":"2024-05-08T21:53:23.574933+0000","last_became_peered":"2024-05-08T21:53:23.574933+0000","last_unstale":"2024-05-08T21:53:23.780009+0000","last_undegraded":"2024-05-08T21:53:23.780009+0000","last_fullsized":"2024-05-08T21:53:23.780009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:04.592572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779970+0000","last_change":"2024-05-08T21:53:23.587183+0000","last_active":"2024-05-08T21:53:23.779970+0000","last_peered":"2024-05-08T21:53:23.779970+0000","last_clean":"2024-05-08T21:53:23.779970+0000","last_became_active":"2024-05-08T21:53:23.586861+0000","last_became_peered":"2024-05-08T21:53:23.586861+0000","last_unstale":"2024-05-08T21:53:23.779970+0000","last_undegraded":"2024-05-08T21:53:23.779970+0000","last_fullsized":"2024-05-08T21:53:23.779970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:48.878049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.587773+0000","last_change":"2024-05-08T21:53:23.587097+0000","last_active":"2024-05-08T21:53:23.587773+0000","last_peered":"2024-05-08T21:53:23.587773+0000","last_clean":"2024-05-08T21:53:23.587773+0000","last_became_active":"2024-05-08T21:53:23.586756+0000","last_became_peered":"2024-05-08T21:53:23.586756+0000","last_unstale":"2024-05-08T21:53:23.587773+0000","last_undegraded":"2024-05-08T21:53:23.587773+0000","last_fullsized":"2024-05-08T21:53:23.587773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:38.938727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566720+0000","last_change":"2024-05-08T21:53:23.581157+0000","last_active":"2024-05-08T21:53:24.566720+0000","last_peered":"2024-05-08T21:53:24.566720+0000","last_clean":"2024-05-08T21:53:24.566720+0000","last_became_active":"2024-05-08T21:53:23.580803+0000","last_became_peered":"2024-05-08T21:53:23.580803+0000","last_unstale":"2024-05-08T21:53:24.566720+0000","last_undegraded":"2024-05-08T21:53:24.566720+0000","last_fullsized":"2024-05-08T21:53:24.566720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:13:59.878096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113677+0000","last_change":"2024-05-08T21:53:23.583093+0000","last_active":"2024-05-08T21:53:25.113677+0000","last_peered":"2024-05-08T21:53:25.113677+0000","last_clean":"2024-05-08T21:53:25.113677+0000","last_became_active":"2024-05-08T21:53:23.582798+0000","last_became_peered":"2024-05-08T21:53:23.582798+0000","last_unstale":"2024-05-08T21:53:25.113677+0000","last_undegraded":"2024-05-08T21:53:25.113677+0000","last_fullsized":"2024-05-08T21:53:25.113677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:53.682817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888997+0000","last_change":"2024-05-08T21:53:23.586322+0000","last_active":"2024-05-08T21:53:25.888997+0000","last_peered":"2024-05-08T21:53:25.888997+0000","last_clean":"2024-05-08T21:53:25.888997+0000","last_became_active":"2024-05-08T21:53:23.585938+0000","last_became_peered":"2024-05-08T21:53:23.585938+0000","last_unstale":"2024-05-08T21:53:25.888997+0000","last_undegraded":"2024-05-08T21:53:25.888997+0000","last_fullsized":"2024-05-08T21:53:25.888997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:22.744048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566197+0000","last_change":"2024-05-08T21:53:23.580496+0000","last_active":"2024-05-08T21:53:24.566197+0000","last_peered":"2024-05-08T21:53:24.566197+0000","last_clean":"2024-05-08T21:53:24.566197+0000","last_became_active":"2024-05-08T21:53:23.580154+0000","last_became_peered":"2024-05-08T21:53:23.580154+0000","last_unstale":"2024-05-08T21:53:24.566197+0000","last_undegraded":"2024-05-08T21:53:24.566197+0000","last_fullsized":"2024-05-08T21:53:24.566197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:50.713758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568936+0000","last_change":"2024-05-08T21:53:23.580549+0000","last_active":"2024-05-08T21:53:24.568936+0000","last_peered":"2024-05-08T21:53:24.568936+0000","last_clean":"2024-05-08T21:53:24.568936+0000","last_became_active":"2024-05-08T21:53:23.579962+0000","last_became_peered":"2024-05-08T21:53:23.579962+0000","last_unstale":"2024-05-08T21:53:24.568936+0000","last_undegraded":"2024-05-08T21:53:24.568936+0000","last_fullsized":"2024-05-08T21:53:24.568936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:52.010295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113920+0000","last_change":"2024-05-08T21:53:23.589640+0000","last_active":"2024-05-08T21:53:25.113920+0000","last_peered":"2024-05-08T21:53:25.113920+0000","last_clean":"2024-05-08T21:53:25.113920+0000","last_became_active":"2024-05-08T21:53:23.589347+0000","last_became_peered":"2024-05-08T21:53:23.589347+0000","last_unstale":"2024-05-08T21:53:25.113920+0000","last_undegraded":"2024-05-08T21:53:25.113920+0000","last_fullsized":"2024-05-08T21:53:25.113920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:40.346414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889087+0000","last_change":"2024-05-08T21:53:23.586430+0000","last_active":"2024-05-08T21:53:25.889087+0000","last_peered":"2024-05-08T21:53:25.889087+0000","last_clean":"2024-05-08T21:53:25.889087+0000","last_became_active":"2024-05-08T21:53:23.585986+0000","last_became_peered":"2024-05-08T21:53:23.585986+0000","last_unstale":"2024-05-08T21:53:25.889087+0000","last_undegraded":"2024-05-08T21:53:25.889087+0000","last_fullsized":"2024-05-08T21:53:25.889087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:09.970481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.586950+0000","last_change":"2024-05-08T21:53:23.585441+0000","last_active":"2024-05-08T21:53:23.586950+0000","last_peered":"2024-05-08T21:53:23.586950+0000","last_clean":"2024-05-08T21:53:23.586950+0000","last_became_active":"2024-05-08T21:53:23.584809+0000","last_became_peered":"2024-05-08T21:53:23.584809+0000","last_unstale":"2024-05-08T21:53:23.586950+0000","last_undegraded":"2024-05-08T21:53:23.586950+0000","last_fullsized":"2024-05-08T21:53:23.586950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:31.599185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806575+0000","last_change":"2024-05-08T21:53:23.575757+0000","last_active":"2024-05-08T21:53:23.806575+0000","last_peered":"2024-05-08T21:53:23.806575+0000","last_clean":"2024-05-08T21:53:23.806575+0000","last_became_active":"2024-05-08T21:53:23.575376+0000","last_became_peered":"2024-05-08T21:53:23.575376+0000","last_unstale":"2024-05-08T21:53:23.806575+0000","last_undegraded":"2024-05-08T21:53:23.806575+0000","last_fullsized":"2024-05-08T21:53:23.806575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.851616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779926+0000","last_change":"2024-05-08T21:53:23.573225+0000","last_active":"2024-05-08T21:53:23.779926+0000","last_peered":"2024-05-08T21:53:23.779926+0000","last_clean":"2024-05-08T21:53:23.779926+0000","last_became_active":"2024-05-08T21:53:23.572784+0000","last_became_peered":"2024-05-08T21:53:23.572784+0000","last_unstale":"2024-05-08T21:53:23.779926+0000","last_undegraded":"2024-05-08T21:53:23.779926+0000","last_fullsized":"2024-05-08T21:53:23.779926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:11.963959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779898+0000","last_change":"2024-05-08T21:53:23.583961+0000","last_active":"2024-05-08T21:53:23.779898+0000","last_peered":"2024-05-08T21:53:23.779898+0000","last_clean":"2024-05-08T21:53:23.779898+0000","last_became_active":"2024-05-08T21:53:23.583532+0000","last_became_peered":"2024-05-08T21:53:23.583532+0000","last_unstale":"2024-05-08T21:53:23.779898+0000","last_undegraded":"2024-05-08T21:53:23.779898+0000","last_fullsized":"2024-05-08T21:53:23.779898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:15:04.318298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568388+0000","last_change":"2024-05-08T21:53:23.577361+0000","last_active":"2024-05-08T21:53:24.568388+0000","last_peered":"2024-05-08T21:53:24.568388+0000","last_clean":"2024-05-08T21:53:24.568388+0000","last_became_active":"2024-05-08T21:53:23.577003+0000","last_became_peered":"2024-05-08T21:53:23.577003+0000","last_unstale":"2024-05-08T21:53:24.568388+0000","last_undegraded":"2024-05-08T21:53:24.568388+0000","last_fullsized":"2024-05-08T21:53:24.568388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:14.617211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703171+0000","last_change":"2024-05-08T21:53:23.579881+0000","last_active":"2024-05-08T21:53:23.703171+0000","last_peered":"2024-05-08T21:53:23.703171+0000","last_clean":"2024-05-08T21:53:23.703171+0000","last_became_active":"2024-05-08T21:53:23.579569+0000","last_became_peered":"2024-05-08T21:53:23.579569+0000","last_unstale":"2024-05-08T21:53:23.703171+0000","last_undegraded":"2024-05-08T21:53:23.703171+0000","last_fullsized":"2024-05-08T21:53:23.703171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:38.210018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.591923+0000","last_change":"2024-05-08T21:53:23.573027+0000","last_active":"2024-05-08T21:53:26.591923+0000","last_peered":"2024-05-08T21:53:26.591923+0000","last_clean":"2024-05-08T21:53:26.591923+0000","last_became_active":"2024-05-08T21:53:23.572709+0000","last_became_peered":"2024-05-08T21:53:23.572709+0000","last_unstale":"2024-05-08T21:53:26.591923+0000","last_undegraded":"2024-05-08T21:53:26.591923+0000","last_fullsized":"2024-05-08T21:53:26.591923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:36.555513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.588568+0000","last_change":"2024-05-08T21:53:23.586364+0000","last_active":"2024-05-08T21:53:23.588568+0000","last_peered":"2024-05-08T21:53:23.588568+0000","last_clean":"2024-05-08T21:53:23.588568+0000","last_became_active":"2024-05-08T21:53:23.585420+0000","last_became_peered":"2024-05-08T21:53:23.585420+0000","last_unstale":"2024-05-08T21:53:23.588568+0000","last_undegraded":"2024-05-08T21:53:23.588568+0000","last_fullsized":"2024-05-08T21:53:23.588568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:01.545821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.578302+0000","last_change":"2024-05-08T21:53:23.148809+0000","last_active":"2024-05-08T21:53:23.578302+0000","last_peered":"2024-05-08T21:53:23.578302+0000","last_clean":"2024-05-08T21:53:23.578302+0000","last_became_active":"2024-05-08T21:53:23.148532+0000","last_became_peered":"2024-05-08T21:53:23.148532+0000","last_unstale":"2024-05-08T21:53:23.578302+0000","last_undegraded":"2024-05-08T21:53:23.578302+0000","last_fullsized":"2024-05-08T21:53:23.578302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:48.209778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566739+0000","last_change":"2024-05-08T21:53:23.584120+0000","last_active":"2024-05-08T21:53:24.566739+0000","last_peered":"2024-05-08T21:53:24.566739+0000","last_clean":"2024-05-08T21:53:24.566739+0000","last_became_active":"2024-05-08T21:53:23.583804+0000","last_became_peered":"2024-05-08T21:53:23.583804+0000","last_unstale":"2024-05-08T21:53:24.566739+0000","last_undegraded":"2024-05-08T21:53:24.566739+0000","last_fullsized":"2024-05-08T21:53:24.566739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:10.669322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.196151+0000","last_change":"2024-05-08T21:53:23.144732+0000","last_active":"2024-05-08T21:53:26.196151+0000","last_peered":"2024-05-08T21:53:26.196151+0000","last_clean":"2024-05-08T21:53:26.196151+0000","last_became_active":"2024-05-08T21:53:23.144405+0000","last_became_peered":"2024-05-08T21:53:23.144405+0000","last_unstale":"2024-05-08T21:53:26.196151+0000","last_undegraded":"2024-05-08T21:53:26.196151+0000","last_fullsized":"2024-05-08T21:53:26.196151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:46.356499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780046+0000","last_change":"2024-05-08T21:53:23.578621+0000","last_active":"2024-05-08T21:53:23.780046+0000","last_peered":"2024-05-08T21:53:23.780046+0000","last_clean":"2024-05-08T21:53:23.780046+0000","last_became_active":"2024-05-08T21:53:23.578331+0000","last_became_peered":"2024-05-08T21:53:23.578331+0000","last_unstale":"2024-05-08T21:53:23.780046+0000","last_undegraded":"2024-05-08T21:53:23.780046+0000","last_fullsized":"2024-05-08T21:53:23.780046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:27.409751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954956+0000","last_change":"2024-05-08T21:53:23.145398+0000","last_active":"2024-05-08T21:53:27.954956+0000","last_peered":"2024-05-08T21:53:27.954956+0000","last_clean":"2024-05-08T21:53:27.954956+0000","last_became_active":"2024-05-08T21:53:23.144997+0000","last_became_peered":"2024-05-08T21:53:23.144997+0000","last_unstale":"2024-05-08T21:53:27.954956+0000","last_undegraded":"2024-05-08T21:53:27.954956+0000","last_fullsized":"2024-05-08T21:53:27.954956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.116791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568566+0000","last_change":"2024-05-08T21:53:23.573118+0000","last_active":"2024-05-08T21:53:24.568566+0000","last_peered":"2024-05-08T21:53:24.568566+0000","last_clean":"2024-05-08T21:53:24.568566+0000","last_became_active":"2024-05-08T21:53:23.572639+0000","last_became_peered":"2024-05-08T21:53:23.572639+0000","last_unstale":"2024-05-08T21:53:24.568566+0000","last_undegraded":"2024-05-08T21:53:24.568566+0000","last_fullsized":"2024-05-08T21:53:24.568566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.786457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.586989+0000","last_change":"2024-05-08T21:53:23.144706+0000","last_active":"2024-05-08T21:53:23.586989+0000","last_peered":"2024-05-08T21:53:23.586989+0000","last_clean":"2024-05-08T21:53:23.586989+0000","last_became_active":"2024-05-08T21:53:23.144380+0000","last_became_peered":"2024-05-08T21:53:23.144380+0000","last_unstale":"2024-05-08T21:53:23.586989+0000","last_undegraded":"2024-05-08T21:53:23.586989+0000","last_fullsized":"2024-05-08T21:53:23.586989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:05.594755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832631+0000","last_change":"2024-05-08T21:53:23.573836+0000","last_active":"2024-05-08T21:53:24.832631+0000","last_peered":"2024-05-08T21:53:24.832631+0000","last_clean":"2024-05-08T21:53:24.832631+0000","last_became_active":"2024-05-08T21:53:23.572323+0000","last_became_peered":"2024-05-08T21:53:23.572323+0000","last_unstale":"2024-05-08T21:53:24.832631+0000","last_undegraded":"2024-05-08T21:53:24.832631+0000","last_fullsized":"2024-05-08T21:53:24.832631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:34.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.948089+0000","last_change":"2024-05-08T21:53:23.147919+0000","last_active":"2024-05-08T21:53:27.948089+0000","last_peered":"2024-05-08T21:53:27.948089+0000","last_clean":"2024-05-08T21:53:27.948089+0000","last_became_active":"2024-05-08T21:53:23.147473+0000","last_became_peered":"2024-05-08T21:53:23.147473+0000","last_unstale":"2024-05-08T21:53:27.948089+0000","last_undegraded":"2024-05-08T21:53:27.948089+0000","last_fullsized":"2024-05-08T21:53:27.948089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:07.158398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566928+0000","last_change":"2024-05-08T21:53:23.576219+0000","last_active":"2024-05-08T21:53:24.566928+0000","last_peered":"2024-05-08T21:53:24.566928+0000","last_clean":"2024-05-08T21:53:24.566928+0000","last_became_active":"2024-05-08T21:53:23.575779+0000","last_became_peered":"2024-05-08T21:53:23.575779+0000","last_unstale":"2024-05-08T21:53:24.566928+0000","last_undegraded":"2024-05-08T21:53:24.566928+0000","last_fullsized":"2024-05-08T21:53:24.566928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:19.813711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953935+0000","last_change":"2024-05-08T21:53:23.152159+0000","last_active":"2024-05-08T21:53:27.953935+0000","last_peered":"2024-05-08T21:53:27.953935+0000","last_clean":"2024-05-08T21:53:27.953935+0000","last_became_active":"2024-05-08T21:53:23.151816+0000","last_became_peered":"2024-05-08T21:53:23.151816+0000","last_unstale":"2024-05-08T21:53:27.953935+0000","last_undegraded":"2024-05-08T21:53:27.953935+0000","last_fullsized":"2024-05-08T21:53:27.953935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.995097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832701+0000","last_change":"2024-05-08T21:53:23.575509+0000","last_active":"2024-05-08T21:53:24.832701+0000","last_peered":"2024-05-08T21:53:24.832701+0000","last_clean":"2024-05-08T21:53:24.832701+0000","last_became_active":"2024-05-08T21:53:23.574966+0000","last_became_peered":"2024-05-08T21:53:23.574966+0000","last_unstale":"2024-05-08T21:53:24.832701+0000","last_undegraded":"2024-05-08T21:53:24.832701+0000","last_fullsized":"2024-05-08T21:53:24.832701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:50.075249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.600882+0000","last_change":"2024-05-08T21:53:23.143151+0000","last_active":"2024-05-08T21:53:27.600882+0000","last_peered":"2024-05-08T21:53:27.600882+0000","last_clean":"2024-05-08T21:53:27.600882+0000","last_became_active":"2024-05-08T21:53:23.142829+0000","last_became_peered":"2024-05-08T21:53:23.142829+0000","last_unstale":"2024-05-08T21:53:27.600882+0000","last_undegraded":"2024-05-08T21:53:27.600882+0000","last_fullsized":"2024-05-08T21:53:27.600882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:49.372293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566098+0000","last_change":"2024-05-08T21:53:23.573587+0000","last_active":"2024-05-08T21:53:24.566098+0000","last_peered":"2024-05-08T21:53:24.566098+0000","last_clean":"2024-05-08T21:53:24.566098+0000","last_became_active":"2024-05-08T21:53:23.573329+0000","last_became_peered":"2024-05-08T21:53:23.573329+0000","last_unstale":"2024-05-08T21:53:24.566098+0000","last_undegraded":"2024-05-08T21:53:24.566098+0000","last_fullsized":"2024-05-08T21:53:24.566098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:49.915501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953090+0000","last_change":"2024-05-08T21:53:23.144147+0000","last_active":"2024-05-08T21:53:27.953090+0000","last_peered":"2024-05-08T21:53:27.953090+0000","last_clean":"2024-05-08T21:53:27.953090+0000","last_became_active":"2024-05-08T21:53:23.143808+0000","last_became_peered":"2024-05-08T21:53:23.143808+0000","last_unstale":"2024-05-08T21:53:27.953090+0000","last_undegraded":"2024-05-08T21:53:27.953090+0000","last_fullsized":"2024-05-08T21:53:27.953090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:07.011887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.592090+0000","last_change":"2024-05-08T21:53:23.583193+0000","last_active":"2024-05-08T21:53:26.592090+0000","last_peered":"2024-05-08T21:53:26.592090+0000","last_clean":"2024-05-08T21:53:26.592090+0000","last_became_active":"2024-05-08T21:53:23.582833+0000","last_became_peered":"2024-05-08T21:53:23.582833+0000","last_unstale":"2024-05-08T21:53:26.592090+0000","last_undegraded":"2024-05-08T21:53:26.592090+0000","last_fullsized":"2024-05-08T21:53:26.592090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:08.708225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.950589+0000","last_change":"2024-05-08T21:53:23.146398+0000","last_active":"2024-05-08T21:53:29.950589+0000","last_peered":"2024-05-08T21:53:29.950589+0000","last_clean":"2024-05-08T21:53:29.950589+0000","last_became_active":"2024-05-08T21:53:23.145823+0000","last_became_peered":"2024-05-08T21:53:23.145823+0000","last_unstale":"2024-05-08T21:53:29.950589+0000","last_undegraded":"2024-05-08T21:53:29.950589+0000","last_fullsized":"2024-05-08T21:53:29.950589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.535292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568901+0000","last_change":"2024-05-08T21:53:23.578664+0000","last_active":"2024-05-08T21:53:24.568901+0000","last_peered":"2024-05-08T21:53:24.568901+0000","last_clean":"2024-05-08T21:53:24.568901+0000","last_became_active":"2024-05-08T21:53:23.578398+0000","last_became_peered":"2024-05-08T21:53:23.578398+0000","last_unstale":"2024-05-08T21:53:24.568901+0000","last_undegraded":"2024-05-08T21:53:24.568901+0000","last_fullsized":"2024-05-08T21:53:24.568901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:10.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.961453+0000","last_change":"2024-05-08T21:53:23.151071+0000","last_active":"2024-05-08T21:53:28.961453+0000","last_peered":"2024-05-08T21:53:28.961453+0000","last_clean":"2024-05-08T21:53:28.961453+0000","last_became_active":"2024-05-08T21:53:23.150566+0000","last_became_peered":"2024-05-08T21:53:23.150566+0000","last_unstale":"2024-05-08T21:53:28.961453+0000","last_undegraded":"2024-05-08T21:53:28.961453+0000","last_fullsized":"2024-05-08T21:53:28.961453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:51.845419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780026+0000","last_change":"2024-05-08T21:53:23.573421+0000","last_active":"2024-05-08T21:53:23.780026+0000","last_peered":"2024-05-08T21:53:23.780026+0000","last_clean":"2024-05-08T21:53:23.780026+0000","last_became_active":"2024-05-08T21:53:23.571240+0000","last_became_peered":"2024-05-08T21:53:23.571240+0000","last_unstale":"2024-05-08T21:53:23.780026+0000","last_undegraded":"2024-05-08T21:53:23.780026+0000","last_fullsized":"2024-05-08T21:53:23.780026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:51.695322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.947244+0000","last_change":"2024-05-08T21:53:23.144552+0000","last_active":"2024-05-08T21:53:27.947244+0000","last_peered":"2024-05-08T21:53:27.947244+0000","last_clean":"2024-05-08T21:53:27.947244+0000","last_became_active":"2024-05-08T21:53:23.144231+0000","last_became_peered":"2024-05-08T21:53:23.144231+0000","last_unstale":"2024-05-08T21:53:27.947244+0000","last_undegraded":"2024-05-08T21:53:27.947244+0000","last_fullsized":"2024-05-08T21:53:27.947244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:44.877035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568575+0000","last_change":"2024-05-08T21:53:23.572913+0000","last_active":"2024-05-08T21:53:24.568575+0000","last_peered":"2024-05-08T21:53:24.568575+0000","last_clean":"2024-05-08T21:53:24.568575+0000","last_became_active":"2024-05-08T21:53:23.572256+0000","last_became_peered":"2024-05-08T21:53:23.572256+0000","last_unstale":"2024-05-08T21:53:24.568575+0000","last_undegraded":"2024-05-08T21:53:24.568575+0000","last_fullsized":"2024-05-08T21:53:24.568575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:05.485571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'13","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.972309+0000","last_change":"2024-05-08T21:53:23.150978+0000","last_active":"2024-05-08T21:53:29.972309+0000","last_peered":"2024-05-08T21:53:29.972309+0000","last_clean":"2024-05-08T21:53:29.972309+0000","last_became_active":"2024-05-08T21:53:23.150637+0000","last_became_peered":"2024-05-08T21:53:23.150637+0000","last_unstale":"2024-05-08T21:53:29.972309+0000","last_undegraded":"2024-05-08T21:53:29.972309+0000","last_fullsized":"2024-05-08T21:53:29.972309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:40.569414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8166,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566921+0000","last_change":"2024-05-08T21:53:23.581145+0000","last_active":"2024-05-08T21:53:24.566921+0000","last_peered":"2024-05-08T21:53:24.566921+0000","last_clean":"2024-05-08T21:53:24.566921+0000","last_became_active":"2024-05-08T21:53:23.580727+0000","last_became_peered":"2024-05-08T21:53:23.580727+0000","last_unstale":"2024-05-08T21:53:24.566921+0000","last_undegraded":"2024-05-08T21:53:24.566921+0000","last_fullsized":"2024-05-08T21:53:24.566921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:58:21.737504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'7","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.944484+0000","last_change":"2024-05-08T21:53:23.146278+0000","last_active":"2024-05-08T21:53:27.944484+0000","last_peered":"2024-05-08T21:53:27.944484+0000","last_clean":"2024-05-08T21:53:27.944484+0000","last_became_active":"2024-05-08T21:53:23.145965+0000","last_became_peered":"2024-05-08T21:53:23.145965+0000","last_unstale":"2024-05-08T21:53:27.944484+0000","last_undegraded":"2024-05-08T21:53:27.944484+0000","last_fullsized":"2024-05-08T21:53:27.944484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:05.370847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806557+0000","last_change":"2024-05-08T21:53:23.576732+0000","last_active":"2024-05-08T21:53:23.806557+0000","last_peered":"2024-05-08T21:53:23.806557+0000","last_clean":"2024-05-08T21:53:23.806557+0000","last_became_active":"2024-05-08T21:53:23.576415+0000","last_became_peered":"2024-05-08T21:53:23.576415+0000","last_unstale":"2024-05-08T21:53:23.806557+0000","last_undegraded":"2024-05-08T21:53:23.806557+0000","last_fullsized":"2024-05-08T21:53:23.806557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:47.184028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953150+0000","last_change":"2024-05-08T21:53:23.148398+0000","last_active":"2024-05-08T21:53:27.953150+0000","last_peered":"2024-05-08T21:53:27.953150+0000","last_clean":"2024-05-08T21:53:27.953150+0000","last_became_active":"2024-05-08T21:53:23.148078+0000","last_became_peered":"2024-05-08T21:53:23.148078+0000","last_unstale":"2024-05-08T21:53:27.953150+0000","last_undegraded":"2024-05-08T21:53:27.953150+0000","last_fullsized":"2024-05-08T21:53:27.953150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:38.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889064+0000","last_change":"2024-05-08T21:53:23.581700+0000","last_active":"2024-05-08T21:53:25.889064+0000","last_peered":"2024-05-08T21:53:25.889064+0000","last_clean":"2024-05-08T21:53:25.889064+0000","last_became_active":"2024-05-08T21:53:23.581346+0000","last_became_peered":"2024-05-08T21:53:23.581346+0000","last_unstale":"2024-05-08T21:53:25.889064+0000","last_undegraded":"2024-05-08T21:53:25.889064+0000","last_fullsized":"2024-05-08T21:53:25.889064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:08.530505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953772+0000","last_change":"2024-05-08T21:53:23.148415+0000","last_active":"2024-05-08T21:53:27.953772+0000","last_peered":"2024-05-08T21:53:27.953772+0000","last_clean":"2024-05-08T21:53:27.953772+0000","last_became_active":"2024-05-08T21:53:23.148037+0000","last_became_peered":"2024-05-08T21:53:23.148037+0000","last_unstale":"2024-05-08T21:53:27.953772+0000","last_undegraded":"2024-05-08T21:53:27.953772+0000","last_fullsized":"2024-05-08T21:53:27.953772+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:17.367009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888950+0000","last_change":"2024-05-08T21:53:23.589596+0000","last_active":"2024-05-08T21:53:25.888950+0000","last_peered":"2024-05-08T21:53:25.888950+0000","last_clean":"2024-05-08T21:53:25.888950+0000","last_became_active":"2024-05-08T21:53:23.589309+0000","last_became_peered":"2024-05-08T21:53:23.589309+0000","last_unstale":"2024-05-08T21:53:25.888950+0000","last_undegraded":"2024-05-08T21:53:25.888950+0000","last_fullsized":"2024-05-08T21:53:25.888950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:41.814532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'16","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.945110+0000","last_change":"2024-05-08T21:53:23.145434+0000","last_active":"2024-05-08T21:53:27.945110+0000","last_peered":"2024-05-08T21:53:27.945110+0000","last_clean":"2024-05-08T21:53:27.945110+0000","last_became_active":"2024-05-08T21:53:23.144328+0000","last_became_peered":"2024-05-08T21:53:23.144328+0000","last_unstale":"2024-05-08T21:53:27.945110+0000","last_undegraded":"2024-05-08T21:53:27.945110+0000","last_fullsized":"2024-05-08T21:53:27.945110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:30.389806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832474+0000","last_change":"2024-05-08T21:53:23.572862+0000","last_active":"2024-05-08T21:53:24.832474+0000","last_peered":"2024-05-08T21:53:24.832474+0000","last_clean":"2024-05-08T21:53:24.832474+0000","last_became_active":"2024-05-08T21:53:23.572516+0000","last_became_peered":"2024-05-08T21:53:23.572516+0000","last_unstale":"2024-05-08T21:53:24.832474+0000","last_undegraded":"2024-05-08T21:53:24.832474+0000","last_fullsized":"2024-05-08T21:53:24.832474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:37.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.567174+0000","last_change":"2024-05-08T21:53:23.146282+0000","last_active":"2024-05-08T21:53:24.567174+0000","last_peered":"2024-05-08T21:53:24.567174+0000","last_clean":"2024-05-08T21:53:24.567174+0000","last_became_active":"2024-05-08T21:53:23.145974+0000","last_became_peered":"2024-05-08T21:53:23.145974+0000","last_unstale":"2024-05-08T21:53:24.567174+0000","last_undegraded":"2024-05-08T21:53:24.567174+0000","last_fullsized":"2024-05-08T21:53:24.567174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:26.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703149+0000","last_change":"2024-05-08T21:53:23.578164+0000","last_active":"2024-05-08T21:53:23.703149+0000","last_peered":"2024-05-08T21:53:23.703149+0000","last_clean":"2024-05-08T21:53:23.703149+0000","last_became_active":"2024-05-08T21:53:23.577832+0000","last_became_peered":"2024-05-08T21:53:23.577832+0000","last_unstale":"2024-05-08T21:53:23.703149+0000","last_undegraded":"2024-05-08T21:53:23.703149+0000","last_fullsized":"2024-05-08T21:53:23.703149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:18.700875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.569471+0000","last_change":"2024-05-08T21:53:23.151368+0000","last_active":"2024-05-08T21:53:23.569471+0000","last_peered":"2024-05-08T21:53:23.569471+0000","last_clean":"2024-05-08T21:53:23.569471+0000","last_became_active":"2024-05-08T21:53:23.151068+0000","last_became_peered":"2024-05-08T21:53:23.151068+0000","last_unstale":"2024-05-08T21:53:23.569471+0000","last_undegraded":"2024-05-08T21:53:23.569471+0000","last_fullsized":"2024-05-08T21:53:23.569471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:42.662149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113749+0000","last_change":"2024-05-08T21:53:23.581387+0000","last_active":"2024-05-08T21:53:25.113749+0000","last_peered":"2024-05-08T21:53:25.113749+0000","last_clean":"2024-05-08T21:53:25.113749+0000","last_became_active":"2024-05-08T21:53:23.579346+0000","last_became_peered":"2024-05-08T21:53:23.579346+0000","last_unstale":"2024-05-08T21:53:25.113749+0000","last_undegraded":"2024-05-08T21:53:25.113749+0000","last_fullsized":"2024-05-08T21:53:25.113749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:42.997082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.971734+0000","last_change":"2024-05-08T21:53:23.150218+0000","last_active":"2024-05-08T21:53:29.971734+0000","last_peered":"2024-05-08T21:53:29.971734+0000","last_clean":"2024-05-08T21:53:29.971734+0000","last_became_active":"2024-05-08T21:53:23.148582+0000","last_became_peered":"2024-05-08T21:53:23.148582+0000","last_unstale":"2024-05-08T21:53:29.971734+0000","last_undegraded":"2024-05-08T21:53:29.971734+0000","last_fullsized":"2024-05-08T21:53:29.971734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:32.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113790+0000","last_change":"2024-05-08T21:53:23.579971+0000","last_active":"2024-05-08T21:53:25.113790+0000","last_peered":"2024-05-08T21:53:25.113790+0000","last_clean":"2024-05-08T21:53:25.113790+0000","last_became_active":"2024-05-08T21:53:23.578267+0000","last_became_peered":"2024-05-08T21:53:23.578267+0000","last_unstale":"2024-05-08T21:53:25.113790+0000","last_undegraded":"2024-05-08T21:53:25.113790+0000","last_fullsized":"2024-05-08T21:53:25.113790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:20.650914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832545+0000","last_change":"2024-05-08T21:53:23.584126+0000","last_active":"2024-05-08T21:53:24.832545+0000","last_peered":"2024-05-08T21:53:24.832545+0000","last_clean":"2024-05-08T21:53:24.832545+0000","last_became_active":"2024-05-08T21:53:23.583801+0000","last_became_peered":"2024-05-08T21:53:23.583801+0000","last_unstale":"2024-05-08T21:53:24.832545+0000","last_undegraded":"2024-05-08T21:53:24.832545+0000","last_fullsized":"2024-05-08T21:53:24.832545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:19.487948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.601829+0000","last_change":"2024-05-08T21:53:23.145221+0000","last_active":"2024-05-08T21:53:27.601829+0000","last_peered":"2024-05-08T21:53:27.601829+0000","last_clean":"2024-05-08T21:53:27.601829+0000","last_became_active":"2024-05-08T21:53:23.144659+0000","last_became_peered":"2024-05-08T21:53:23.144659+0000","last_unstale":"2024-05-08T21:53:27.601829+0000","last_undegraded":"2024-05-08T21:53:27.601829+0000","last_fullsized":"2024-05-08T21:53:27.601829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:58.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566506+0000","last_change":"2024-05-08T21:53:23.581748+0000","last_active":"2024-05-08T21:53:24.566506+0000","last_peered":"2024-05-08T21:53:24.566506+0000","last_clean":"2024-05-08T21:53:24.566506+0000","last_became_active":"2024-05-08T21:53:23.581336+0000","last_became_peered":"2024-05-08T21:53:23.581336+0000","last_unstale":"2024-05-08T21:53:24.566506+0000","last_undegraded":"2024-05-08T21:53:24.566506+0000","last_fullsized":"2024-05-08T21:53:24.566506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:31.158473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954240+0000","last_change":"2024-05-08T21:53:23.147073+0000","last_active":"2024-05-08T21:53:27.954240+0000","last_peered":"2024-05-08T21:53:27.954240+0000","last_clean":"2024-05-08T21:53:27.954240+0000","last_became_active":"2024-05-08T21:53:23.146720+0000","last_became_peered":"2024-05-08T21:53:23.146720+0000","last_unstale":"2024-05-08T21:53:27.954240+0000","last_undegraded":"2024-05-08T21:53:27.954240+0000","last_fullsized":"2024-05-08T21:53:27.954240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:40.566772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":321,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.573447+0000","last_change":"2024-05-08T21:52:33.304674+0000","last_active":"2024-05-08T21:53:23.573447+0000","last_peered":"2024-05-08T21:53:23.573447+0000","last_clean":"2024-05-08T21:53:23.573447+0000","last_became_active":"2024-05-08T21:52:33.304193+0000","last_became_peered":"2024-05-08T21:52:33.304193+0000","last_unstale":"2024-05-08T21:53:23.573447+0000","last_undegraded":"2024-05-08T21:53:23.573447+0000","last_fullsized":"2024-05-08T21:53:23.573447+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_clean_scrub_stamp":"2024-05-08T21:50:08.558194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:21.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806496+0000","last_change":"2024-05-08T21:53:23.583793+0000","last_active":"2024-05-08T21:53:23.806496+0000","last_peered":"2024-05-08T21:53:23.806496+0000","last_clean":"2024-05-08T21:53:23.806496+0000","last_became_active":"2024-05-08T21:53:23.583464+0000","last_became_peered":"2024-05-08T21:53:23.583464+0000","last_unstale":"2024-05-08T21:53:23.806496+0000","last_undegraded":"2024-05-08T21:53:23.806496+0000","last_fullsized":"2024-05-08T21:53:23.806496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:02.106595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.600923+0000","last_change":"2024-05-08T21:53:23.150460+0000","last_active":"2024-05-08T21:53:27.600923+0000","last_peered":"2024-05-08T21:53:27.600923+0000","last_clean":"2024-05-08T21:53:27.600923+0000","last_became_active":"2024-05-08T21:53:23.150025+0000","last_became_peered":"2024-05-08T21:53:23.150025+0000","last_unstale":"2024-05-08T21:53:27.600923+0000","last_undegraded":"2024-05-08T21:53:27.600923+0000","last_fullsized":"2024-05-08T21:53:27.600923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:28:09.311683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954703+0000","last_change":"2024-05-08T21:53:23.143160+0000","last_active":"2024-05-08T21:53:27.954703+0000","last_peered":"2024-05-08T21:53:27.954703+0000","last_clean":"2024-05-08T21:53:27.954703+0000","last_became_active":"2024-05-08T21:53:23.142718+0000","last_became_peered":"2024-05-08T21:53:23.142718+0000","last_unstale":"2024-05-08T21:53:27.954703+0000","last_undegraded":"2024-05-08T21:53:27.954703+0000","last_fullsized":"2024-05-08T21:53:27.954703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:54.148427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806605+0000","last_change":"2024-05-08T21:53:23.587069+0000","last_active":"2024-05-08T21:53:23.806605+0000","last_peered":"2024-05-08T21:53:23.806605+0000","last_clean":"2024-05-08T21:53:23.806605+0000","last_became_active":"2024-05-08T21:53:23.586782+0000","last_became_peered":"2024-05-08T21:53:23.586782+0000","last_unstale":"2024-05-08T21:53:23.806605+0000","last_undegraded":"2024-05-08T21:53:23.806605+0000","last_fullsized":"2024-05-08T21:53:23.806605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:08.897700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.567170+0000","last_change":"2024-05-08T21:53:23.142641+0000","last_active":"2024-05-08T21:53:23.567170+0000","last_peered":"2024-05-08T21:53:23.567170+0000","last_clean":"2024-05-08T21:53:23.567170+0000","last_became_active":"2024-05-08T21:53:23.142294+0000","last_became_peered":"2024-05-08T21:53:23.142294+0000","last_unstale":"2024-05-08T21:53:23.567170+0000","last_undegraded":"2024-05-08T21:53:23.567170+0000","last_fullsized":"2024-05-08T21:53:23.567170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:11.687406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566396+0000","last_change":"2024-05-08T21:53:23.580729+0000","last_active":"2024-05-08T21:53:24.566396+0000","last_peered":"2024-05-08T21:53:24.566396+0000","last_clean":"2024-05-08T21:53:24.566396+0000","last_became_active":"2024-05-08T21:53:23.580159+0000","last_became_peered":"2024-05-08T21:53:23.580159+0000","last_unstale":"2024-05-08T21:53:24.566396+0000","last_undegraded":"2024-05-08T21:53:24.566396+0000","last_fullsized":"2024-05-08T21:53:24.566396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:43:33.336308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.597286+0000","last_change":"2024-05-08T21:53:23.144242+0000","last_active":"2024-05-08T21:53:26.597286+0000","last_peered":"2024-05-08T21:53:26.597286+0000","last_clean":"2024-05-08T21:53:26.597286+0000","last_became_active":"2024-05-08T21:53:23.143905+0000","last_became_peered":"2024-05-08T21:53:23.143905+0000","last_unstale":"2024-05-08T21:53:26.597286+0000","last_undegraded":"2024-05-08T21:53:26.597286+0000","last_fullsized":"2024-05-08T21:53:26.597286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:41.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779991+0000","last_change":"2024-05-08T21:53:23.576972+0000","last_active":"2024-05-08T21:53:23.779991+0000","last_peered":"2024-05-08T21:53:23.779991+0000","last_clean":"2024-05-08T21:53:23.779991+0000","last_became_active":"2024-05-08T21:53:23.576668+0000","last_became_peered":"2024-05-08T21:53:23.576668+0000","last_unstale":"2024-05-08T21:53:23.779991+0000","last_undegraded":"2024-05-08T21:53:23.779991+0000","last_fullsized":"2024-05-08T21:53:23.779991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:25.545108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.595186+0000","last_change":"2024-05-08T21:53:23.145610+0000","last_active":"2024-05-08T21:53:26.595186+0000","last_peered":"2024-05-08T21:53:26.595186+0000","last_clean":"2024-05-08T21:53:26.595186+0000","last_became_active":"2024-05-08T21:53:23.145075+0000","last_became_peered":"2024-05-08T21:53:23.145075+0000","last_unstale":"2024-05-08T21:53:26.595186+0000","last_undegraded":"2024-05-08T21:53:26.595186+0000","last_fullsized":"2024-05-08T21:53:26.595186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:58.351303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566650+0000","last_change":"2024-05-08T21:53:23.579335+0000","last_active":"2024-05-08T21:53:24.566650+0000","last_peered":"2024-05-08T21:53:24.566650+0000","last_clean":"2024-05-08T21:53:24.566650+0000","last_became_active":"2024-05-08T21:53:23.578956+0000","last_became_peered":"2024-05-08T21:53:23.578956+0000","last_unstale":"2024-05-08T21:53:24.566650+0000","last_undegraded":"2024-05-08T21:53:24.566650+0000","last_fullsized":"2024-05-08T21:53:24.566650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:34.872954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.964020+0000","last_change":"2024-05-08T21:53:23.144108+0000","last_active":"2024-05-08T21:53:28.964020+0000","last_peered":"2024-05-08T21:53:28.964020+0000","last_clean":"2024-05-08T21:53:28.964020+0000","last_became_active":"2024-05-08T21:53:23.143786+0000","last_became_peered":"2024-05-08T21:53:23.143786+0000","last_unstale":"2024-05-08T21:53:28.964020+0000","last_undegraded":"2024-05-08T21:53:28.964020+0000","last_fullsized":"2024-05-08T21:53:28.964020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:15.512879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703096+0000","last_change":"2024-05-08T21:53:23.587667+0000","last_active":"2024-05-08T21:53:23.703096+0000","last_peered":"2024-05-08T21:53:23.703096+0000","last_clean":"2024-05-08T21:53:23.703096+0000","last_became_active":"2024-05-08T21:53:23.586908+0000","last_became_peered":"2024-05-08T21:53:23.586908+0000","last_unstale":"2024-05-08T21:53:23.703096+0000","last_undegraded":"2024-05-08T21:53:23.703096+0000","last_fullsized":"2024-05-08T21:53:23.703096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:06.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.601100+0000","last_change":"2024-05-08T21:53:23.150602+0000","last_active":"2024-05-08T21:53:24.601100+0000","last_peered":"2024-05-08T21:53:24.601100+0000","last_clean":"2024-05-08T21:53:24.601100+0000","last_became_active":"2024-05-08T21:53:23.150256+0000","last_became_peered":"2024-05-08T21:53:23.150256+0000","last_unstale":"2024-05-08T21:53:24.601100+0000","last_undegraded":"2024-05-08T21:53:24.601100+0000","last_fullsized":"2024-05-08T21:53:24.601100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:22:05.088246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566829+0000","last_change":"2024-05-08T21:53:23.577768+0000","last_active":"2024-05-08T21:53:24.566829+0000","last_peered":"2024-05-08T21:53:24.566829+0000","last_clean":"2024-05-08T21:53:24.566829+0000","last_became_active":"2024-05-08T21:53:23.577388+0000","last_became_peered":"2024-05-08T21:53:23.577388+0000","last_unstale":"2024-05-08T21:53:24.566829+0000","last_undegraded":"2024-05-08T21:53:24.566829+0000","last_fullsized":"2024-05-08T21:53:24.566829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:24.707904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.593957+0000","last_change":"2024-05-08T21:53:23.145165+0000","last_active":"2024-05-08T21:53:26.593957+0000","last_peered":"2024-05-08T21:53:26.593957+0000","last_clean":"2024-05-08T21:53:26.593957+0000","last_became_active":"2024-05-08T21:53:23.144465+0000","last_became_peered":"2024-05-08T21:53:23.144465+0000","last_unstale":"2024-05-08T21:53:26.593957+0000","last_undegraded":"2024-05-08T21:53:26.593957+0000","last_fullsized":"2024-05-08T21:53:26.593957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:47.229905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889021+0000","last_change":"2024-05-08T21:53:23.583476+0000","last_active":"2024-05-08T21:53:25.889021+0000","last_peered":"2024-05-08T21:53:25.889021+0000","last_clean":"2024-05-08T21:53:25.889021+0000","last_became_active":"2024-05-08T21:53:23.582989+0000","last_became_peered":"2024-05-08T21:53:23.582989+0000","last_unstale":"2024-05-08T21:53:25.889021+0000","last_undegraded":"2024-05-08T21:53:25.889021+0000","last_fullsized":"2024-05-08T21:53:25.889021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:05.402755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953612+0000","last_change":"2024-05-08T21:53:23.147469+0000","last_active":"2024-05-08T21:53:27.953612+0000","last_peered":"2024-05-08T21:53:27.953612+0000","last_clean":"2024-05-08T21:53:27.953612+0000","last_became_active":"2024-05-08T21:53:23.147145+0000","last_became_peered":"2024-05-08T21:53:23.147145+0000","last_unstale":"2024-05-08T21:53:27.953612+0000","last_undegraded":"2024-05-08T21:53:27.953612+0000","last_fullsized":"2024-05-08T21:53:27.953612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:15.806118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889101+0000","last_change":"2024-05-08T21:53:23.583221+0000","last_active":"2024-05-08T21:53:25.889101+0000","last_peered":"2024-05-08T21:53:25.889101+0000","last_clean":"2024-05-08T21:53:25.889101+0000","last_became_active":"2024-05-08T21:53:23.582942+0000","last_became_peered":"2024-05-08T21:53:23.582942+0000","last_unstale":"2024-05-08T21:53:25.889101+0000","last_undegraded":"2024-05-08T21:53:25.889101+0000","last_fullsized":"2024-05-08T21:53:25.889101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:32.188611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953180+0000","last_change":"2024-05-08T21:53:23.143721+0000","last_active":"2024-05-08T21:53:27.953180+0000","last_peered":"2024-05-08T21:53:27.953180+0000","last_clean":"2024-05-08T21:53:27.953180+0000","last_became_active":"2024-05-08T21:53:23.143227+0000","last_became_peered":"2024-05-08T21:53:23.143227+0000","last_unstale":"2024-05-08T21:53:27.953180+0000","last_undegraded":"2024-05-08T21:53:27.953180+0000","last_fullsized":"2024-05-08T21:53:27.953180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:09.619865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889075+0000","last_change":"2024-05-08T21:53:23.585820+0000","last_active":"2024-05-08T21:53:25.889075+0000","last_peered":"2024-05-08T21:53:25.889075+0000","last_clean":"2024-05-08T21:53:25.889075+0000","last_became_active":"2024-05-08T21:53:23.585435+0000","last_became_peered":"2024-05-08T21:53:23.585435+0000","last_unstale":"2024-05-08T21:53:25.889075+0000","last_undegraded":"2024-05-08T21:53:25.889075+0000","last_fullsized":"2024-05-08T21:53:25.889075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:48.696651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.185531+0000","last_change":"2024-05-08T21:53:23.141992+0000","last_active":"2024-05-08T21:53:26.185531+0000","last_peered":"2024-05-08T21:53:26.185531+0000","last_clean":"2024-05-08T21:53:26.185531+0000","last_became_active":"2024-05-08T21:53:23.141670+0000","last_became_peered":"2024-05-08T21:53:23.141670+0000","last_unstale":"2024-05-08T21:53:26.185531+0000","last_undegraded":"2024-05-08T21:53:26.185531+0000","last_fullsized":"2024-05-08T21:53:26.185531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:53.191681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889139+0000","last_change":"2024-05-08T21:53:23.586649+0000","last_active":"2024-05-08T21:53:25.889139+0000","last_peered":"2024-05-08T21:53:25.889139+0000","last_clean":"2024-05-08T21:53:25.889139+0000","last_became_active":"2024-05-08T21:53:23.586069+0000","last_became_peered":"2024-05-08T21:53:23.586069+0000","last_unstale":"2024-05-08T21:53:25.889139+0000","last_undegraded":"2024-05-08T21:53:25.889139+0000","last_fullsized":"2024-05-08T21:53:25.889139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:59.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703112+0000","last_change":"2024-05-08T21:53:23.588268+0000","last_active":"2024-05-08T21:53:23.703112+0000","last_peered":"2024-05-08T21:53:23.703112+0000","last_clean":"2024-05-08T21:53:23.703112+0000","last_became_active":"2024-05-08T21:53:23.587858+0000","last_became_peered":"2024-05-08T21:53:23.587858+0000","last_unstale":"2024-05-08T21:53:23.703112+0000","last_undegraded":"2024-05-08T21:53:23.703112+0000","last_fullsized":"2024-05-08T21:53:23.703112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:47.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.589133+0000","last_change":"2024-05-08T21:53:23.587227+0000","last_active":"2024-05-08T21:53:23.589133+0000","last_peered":"2024-05-08T21:53:23.589133+0000","last_clean":"2024-05-08T21:53:23.589133+0000","last_became_active":"2024-05-08T21:53:23.586807+0000","last_became_peered":"2024-05-08T21:53:23.586807+0000","last_unstale":"2024-05-08T21:53:23.589133+0000","last_undegraded":"2024-05-08T21:53:23.589133+0000","last_fullsized":"2024-05-08T21:53:23.589133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:29.814057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585531+0000","last_change":"2024-05-08T21:53:23.583968+0000","last_active":"2024-05-08T21:53:23.585531+0000","last_peered":"2024-05-08T21:53:23.585531+0000","last_clean":"2024-05-08T21:53:23.585531+0000","last_became_active":"2024-05-08T21:53:23.583625+0000","last_became_peered":"2024-05-08T21:53:23.583625+0000","last_unstale":"2024-05-08T21:53:23.585531+0000","last_undegraded":"2024-05-08T21:53:23.585531+0000","last_fullsized":"2024-05-08T21:53:23.585531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:40.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779949+0000","last_change":"2024-05-08T21:53:23.571284+0000","last_active":"2024-05-08T21:53:23.779949+0000","last_peered":"2024-05-08T21:53:23.779949+0000","last_clean":"2024-05-08T21:53:23.779949+0000","last_became_active":"2024-05-08T21:53:23.570915+0000","last_became_peered":"2024-05-08T21:53:23.570915+0000","last_unstale":"2024-05-08T21:53:23.779949+0000","last_undegraded":"2024-05-08T21:53:23.779949+0000","last_fullsized":"2024-05-08T21:53:23.779949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:52.562178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806616+0000","last_change":"2024-05-08T21:53:23.572011+0000","last_active":"2024-05-08T21:53:23.806616+0000","last_peered":"2024-05-08T21:53:23.806616+0000","last_clean":"2024-05-08T21:53:23.806616+0000","last_became_active":"2024-05-08T21:53:23.571788+0000","last_became_peered":"2024-05-08T21:53:23.571788+0000","last_unstale":"2024-05-08T21:53:23.806616+0000","last_undegraded":"2024-05-08T21:53:23.806616+0000","last_fullsized":"2024-05-08T21:53:23.806616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:43.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703137+0000","last_change":"2024-05-08T21:53:23.582823+0000","last_active":"2024-05-08T21:53:23.703137+0000","last_peered":"2024-05-08T21:53:23.703137+0000","last_clean":"2024-05-08T21:53:23.703137+0000","last_became_active":"2024-05-08T21:53:23.582366+0000","last_became_peered":"2024-05-08T21:53:23.582366+0000","last_unstale":"2024-05-08T21:53:23.703137+0000","last_undegraded":"2024-05-08T21:53:23.703137+0000","last_fullsized":"2024-05-08T21:53:23.703137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:44.029915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585670+0000","last_change":"2024-05-08T21:53:23.583566+0000","last_active":"2024-05-08T21:53:23.585670+0000","last_peered":"2024-05-08T21:53:23.585670+0000","last_clean":"2024-05-08T21:53:23.585670+0000","last_became_active":"2024-05-08T21:53:23.583204+0000","last_became_peered":"2024-05-08T21:53:23.583204+0000","last_unstale":"2024-05-08T21:53:23.585670+0000","last_undegraded":"2024-05-08T21:53:23.585670+0000","last_fullsized":"2024-05-08T21:53:23.585670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:41.426005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566280+0000","last_change":"2024-05-08T21:53:23.587105+0000","last_active":"2024-05-08T21:53:24.566280+0000","last_peered":"2024-05-08T21:53:24.566280+0000","last_clean":"2024-05-08T21:53:24.566280+0000","last_became_active":"2024-05-08T21:53:23.586773+0000","last_became_peered":"2024-05-08T21:53:23.586773+0000","last_unstale":"2024-05-08T21:53:24.566280+0000","last_undegraded":"2024-05-08T21:53:24.566280+0000","last_fullsized":"2024-05-08T21:53:24.566280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:50.809186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832696+0000","last_change":"2024-05-08T21:53:23.576779+0000","last_active":"2024-05-08T21:53:24.832696+0000","last_peered":"2024-05-08T21:53:24.832696+0000","last_clean":"2024-05-08T21:53:24.832696+0000","last_became_active":"2024-05-08T21:53:23.576434+0000","last_became_peered":"2024-05-08T21:53:23.576434+0000","last_unstale":"2024-05-08T21:53:24.832696+0000","last_undegraded":"2024-05-08T21:53:24.832696+0000","last_fullsized":"2024-05-08T21:53:24.832696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:10.168522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566243+0000","last_change":"2024-05-08T21:53:23.585196+0000","last_active":"2024-05-08T21:53:24.566243+0000","last_peered":"2024-05-08T21:53:24.566243+0000","last_clean":"2024-05-08T21:53:24.566243+0000","last_became_active":"2024-05-08T21:53:23.584160+0000","last_became_peered":"2024-05-08T21:53:23.584160+0000","last_unstale":"2024-05-08T21:53:24.566243+0000","last_undegraded":"2024-05-08T21:53:24.566243+0000","last_fullsized":"2024-05-08T21:53:24.566243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:15.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568545+0000","last_change":"2024-05-08T21:53:23.577887+0000","last_active":"2024-05-08T21:53:24.568545+0000","last_peered":"2024-05-08T21:53:24.568545+0000","last_clean":"2024-05-08T21:53:24.568545+0000","last_became_active":"2024-05-08T21:53:23.577597+0000","last_became_peered":"2024-05-08T21:53:23.577597+0000","last_unstale":"2024-05-08T21:53:24.568545+0000","last_undegraded":"2024-05-08T21:53:24.568545+0000","last_fullsized":"2024-05-08T21:53:24.568545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:33.704002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806594+0000","last_change":"2024-05-08T21:53:23.575180+0000","last_active":"2024-05-08T21:53:23.806594+0000","last_peered":"2024-05-08T21:53:23.806594+0000","last_clean":"2024-05-08T21:53:23.806594+0000","last_became_active":"2024-05-08T21:53:23.574521+0000","last_became_peered":"2024-05-08T21:53:23.574521+0000","last_unstale":"2024-05-08T21:53:23.806594+0000","last_undegraded":"2024-05-08T21:53:23.806594+0000","last_fullsized":"2024-05-08T21:53:23.806594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:59.609492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703125+0000","last_change":"2024-05-08T21:53:23.585380+0000","last_active":"2024-05-08T21:53:23.703125+0000","last_peered":"2024-05-08T21:53:23.703125+0000","last_clean":"2024-05-08T21:53:23.703125+0000","last_became_active":"2024-05-08T21:53:23.585073+0000","last_became_peered":"2024-05-08T21:53:23.585073+0000","last_unstale":"2024-05-08T21:53:23.703125+0000","last_undegraded":"2024-05-08T21:53:23.703125+0000","last_fullsized":"2024-05-08T21:53:23.703125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:33.572410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13971,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":36,"num_read_kb":29,"num_write":161,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":58587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":860160,"data_stored":410274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":10,"up_from":67,"seq":287762808847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1495040,"data_stored":1003205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.44}]},{"osd":3,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1,"5min":1,"15min":1},"last":0.502}]},{"osd":4,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.72799999999999998}]},{"osd":9,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.73999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972369,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":401142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.377}]},{"osd":1,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.65200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":8,"up_from":55,"seq":236223201301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":856064,"data_stored":409425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005}]},{"osd":3,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.83199999999999996}]},{"osd":5,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.624},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.85499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1474560,"data_stored":1003675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33800000000000002}]},{"osd":2,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.498}]},{"osd":4,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.76800000000000002}]},{"osd":6,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47899999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.749}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58699999999999997}]},{"osd":10,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":409242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.28899999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.432},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.437},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.871,"15min":0.871},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed May 8 21:52:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.748},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71899999999999997}]},{"osd":8,"last update":"Wed May 8 21:52:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70599999999999996}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.5},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.78500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed May 8 21:53:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.749,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.78400000000000003}]},{"osd":3,"last update":"Wed May 8 21:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed May 8 21:52:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64700000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.33700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.748},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.83999999999999997}]},{"osd":6,"last update":"Wed May 8 21:53:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.80200000000000005}]},{"osd":8,"last update":"Wed May 8 21:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.94299999999999995}]},{"osd":9,"last update":"Wed May 8 21:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.85399999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.92100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":411148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.498,"15min":0.46899999999999997},"min":{"1min":0.44600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48799999999999999,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.496}]},{"osd":2,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.83599999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed May 8 21:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed May 8 21:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Wed May 8 21:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed May 8 21:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73199999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":411996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48999999999999999,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed May 8 21:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82099999999999995}]},{"osd":5,"last update":"Wed May 8 21:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed May 8 21:53:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71999999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82799999999999996,"5min":0.879,"15min":0.879},"last":0.75700000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52900000000000003}]},{"osd":8,"last update":"Wed May 8 21:52:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.80300000000000005}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]}]},{"osd":3,"up_from":24,"seq":103079215144,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":409677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.34699999999999998},"min":{"1min":0.28999999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.629},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.375,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76500000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed May 8 21:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.502}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed May 8 21:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.627},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.48599999999999999}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.877}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":409748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.377,"15min":0.377},"max":{"1min":0.76800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.755}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.77300000000000002}]},{"osd":7,"last update":"Wed May 8 21:52:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.875},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.84499999999999997}]},{"osd":8,"last update":"Wed May 8 21:52:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91400000000000003}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.0109999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1482752,"data_stored":1001939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.77000000000000002}]},{"osd":1,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.37,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.70399999999999996}]},{"osd":2,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.253,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.441},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48699999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.78100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59899999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84499999999999997}]},{"osd":6,"last update":"Wed May 8 21:53:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55900000000000005}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.78500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":32107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":10247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:53:32.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 -- 172.21.15.64:0/2426308189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104210 msgr2=0x7f6050106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 --2- 172.21.15.64:0/2426308189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104210 0x7f6050106600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6044009990 tx=0x7f6044009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 -- 172.21.15.64:0/2426308189 shutdown_connections 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 --2- 172.21.15.64:0/2426308189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60501018e0 0x7f6050103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 --2- 172.21.15.64:0/2426308189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6050106b40 0x7f6050108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 --2- 172.21.15.64:0/2426308189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104210 0x7f6050106600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 -- 172.21.15.64:0/2426308189 >> 172.21.15.64:0/2426308189 conn(0x7f60500fb2f0 msgr2=0x7f60500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 -- 172.21.15.64:0/2426308189 shutdown_connections 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.980+0000 7f6055424700 1 -- 172.21.15.64:0/2426308189 wait complete. 2024-05-08T21:53:32.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 Processor -- start 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 -- start start 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 0x7f6050196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6050104210 0x7f6050197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6050106b40 0x7f605019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605019e830 con 0x7f60501018e0 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605019e9a0 con 0x7f6050104210 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f6055424700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605019eb60 con 0x7f6050106b40 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f604effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 0x7f6050196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f604effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 0x7f6050196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59756/0 (socket says 172.21.15.64:59756) 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.981+0000 7f604effd700 1 -- 172.21.15.64:0/222508814 learned_addr learned my addr 172.21.15.64:0/222508814 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604e7fc700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6050104210 0x7f6050197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604f7fe700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6050106b40 0x7f605019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604effd700 1 -- 172.21.15.64:0/222508814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6050106b40 msgr2=0x7f605019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604effd700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6050106b40 0x7f605019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604effd700 1 -- 172.21.15.64:0/222508814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6050104210 msgr2=0x7f6050197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604effd700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6050104210 0x7f6050197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604effd700 1 -- 172.21.15.64:0/222508814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440095f0 con 0x7f60501018e0 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f604effd700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 0x7f6050196d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f604000c9b0 tx=0x7f604000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040007b00 con 0x7f60501018e0 2024-05-08T21:53:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605019ee40 con 0x7f60501018e0 2024-05-08T21:53:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604000ce50 con 0x7f60501018e0 2024-05-08T21:53:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605019f270 con 0x7f60501018e0 2024-05-08T21:53:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.982+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040018660 con 0x7f60501018e0 2024-05-08T21:53:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.983+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60500fce70 con 0x7f60501018e0 2024-05-08T21:53:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.984+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f604000f450 con 0x7f60501018e0 2024-05-08T21:53:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.985+0000 7f6037fff700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f60380a11b0 0x7f60380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.985+0000 7f604e7fc700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f60380a11b0 0x7f60380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.985+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f60400cc7b0 con 0x7f60501018e0 2024-05-08T21:53:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.985+0000 7f604e7fc700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f60380a11b0 0x7f60380a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f604400b5c0 tx=0x7f6044003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:32.989+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6040095f90 con 0x7f60501018e0 2024-05-08T21:53:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.191+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f605002ce10 con 0x7f60380a11b0 2024-05-08T21:53:33.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.200+0000 7f6037fff700 1 -- 172.21.15.64:0/222508814 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332512 (secure 0 0 0) 0x7f605002ce10 con 0x7f60380a11b0 2024-05-08T21:53:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f60380a11b0 msgr2=0x7f60380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f60380a11b0 0x7f60380a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f604400b5c0 tx=0x7f6044003790 comp rx=0 tx=0).stop 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 msgr2=0x7f6050196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 0x7f6050196d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f604000c9b0 tx=0x7f604000ccc0 comp rx=0 tx=0).stop 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 shutdown_connections 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f60380a11b0 0x7f60380a3670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6050106b40 0x7f605019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60501018e0 0x7f6050196d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 --2- 172.21.15.64:0/222508814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6050104210 0x7f6050197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.204+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 >> 172.21.15.64:0/222508814 conn(0x7f60500fb2f0 msgr2=0x7f6050102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.205+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 shutdown_connections 2024-05-08T21:53:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:33.205+0000 7f6055424700 1 -- 172.21.15.64:0/222508814 wait complete. 2024-05-08T21:53:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-08T21:53:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:32 smithi064 ceph-mon[42500]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:32 smithi064 ceph-mon[42500]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:33.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:32 smithi064 ceph-mon[42500]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T21:53:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:32 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/779800366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:33.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:32 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2837927255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:33.218 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-05-08T21:53:32.095930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604339,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":162,"num_read_kb":138,"num_write":342,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356672,"kb_used_data":11976,"kb_used_omap":18,"kb_used_meta":344557,"kb_avail":1124585152,"statfs":{"total":1151940427776,"available":1151575195648,"internally_reserved":0,"allocated":12263424,"data_stored":6713547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":352826693},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13971,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":36,"num_read_kb":29,"num_write":161,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.538118"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703161+0000","last_change":"2024-05-08T21:53:23.579256+0000","last_active":"2024-05-08T21:53:23.703161+0000","last_peered":"2024-05-08T21:53:23.703161+0000","last_clean":"2024-05-08T21:53:23.703161+0000","last_became_active":"2024-05-08T21:53:23.578867+0000","last_became_peered":"2024-05-08T21:53:23.578867+0000","last_unstale":"2024-05-08T21:53:23.703161+0000","last_undegraded":"2024-05-08T21:53:23.703161+0000","last_fullsized":"2024-05-08T21:53:23.703161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:07.123241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780009+0000","last_change":"2024-05-08T21:53:23.575229+0000","last_active":"2024-05-08T21:53:23.780009+0000","last_peered":"2024-05-08T21:53:23.780009+0000","last_clean":"2024-05-08T21:53:23.780009+0000","last_became_active":"2024-05-08T21:53:23.574933+0000","last_became_peered":"2024-05-08T21:53:23.574933+0000","last_unstale":"2024-05-08T21:53:23.780009+0000","last_undegraded":"2024-05-08T21:53:23.780009+0000","last_fullsized":"2024-05-08T21:53:23.780009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:04.592572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779970+0000","last_change":"2024-05-08T21:53:23.587183+0000","last_active":"2024-05-08T21:53:23.779970+0000","last_peered":"2024-05-08T21:53:23.779970+0000","last_clean":"2024-05-08T21:53:23.779970+0000","last_became_active":"2024-05-08T21:53:23.586861+0000","last_became_peered":"2024-05-08T21:53:23.586861+0000","last_unstale":"2024-05-08T21:53:23.779970+0000","last_undegraded":"2024-05-08T21:53:23.779970+0000","last_fullsized":"2024-05-08T21:53:23.779970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:48.878049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.587773+0000","last_change":"2024-05-08T21:53:23.587097+0000","last_active":"2024-05-08T21:53:23.587773+0000","last_peered":"2024-05-08T21:53:23.587773+0000","last_clean":"2024-05-08T21:53:23.587773+0000","last_became_active":"2024-05-08T21:53:23.586756+0000","last_became_peered":"2024-05-08T21:53:23.586756+0000","last_unstale":"2024-05-08T21:53:23.587773+0000","last_undegraded":"2024-05-08T21:53:23.587773+0000","last_fullsized":"2024-05-08T21:53:23.587773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:38.938727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566720+0000","last_change":"2024-05-08T21:53:23.581157+0000","last_active":"2024-05-08T21:53:24.566720+0000","last_peered":"2024-05-08T21:53:24.566720+0000","last_clean":"2024-05-08T21:53:24.566720+0000","last_became_active":"2024-05-08T21:53:23.580803+0000","last_became_peered":"2024-05-08T21:53:23.580803+0000","last_unstale":"2024-05-08T21:53:24.566720+0000","last_undegraded":"2024-05-08T21:53:24.566720+0000","last_fullsized":"2024-05-08T21:53:24.566720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:13:59.878096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113677+0000","last_change":"2024-05-08T21:53:23.583093+0000","last_active":"2024-05-08T21:53:25.113677+0000","last_peered":"2024-05-08T21:53:25.113677+0000","last_clean":"2024-05-08T21:53:25.113677+0000","last_became_active":"2024-05-08T21:53:23.582798+0000","last_became_peered":"2024-05-08T21:53:23.582798+0000","last_unstale":"2024-05-08T21:53:25.113677+0000","last_undegraded":"2024-05-08T21:53:25.113677+0000","last_fullsized":"2024-05-08T21:53:25.113677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:53.682817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888997+0000","last_change":"2024-05-08T21:53:23.586322+0000","last_active":"2024-05-08T21:53:25.888997+0000","last_peered":"2024-05-08T21:53:25.888997+0000","last_clean":"2024-05-08T21:53:25.888997+0000","last_became_active":"2024-05-08T21:53:23.585938+0000","last_became_peered":"2024-05-08T21:53:23.585938+0000","last_unstale":"2024-05-08T21:53:25.888997+0000","last_undegraded":"2024-05-08T21:53:25.888997+0000","last_fullsized":"2024-05-08T21:53:25.888997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:22.744048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566197+0000","last_change":"2024-05-08T21:53:23.580496+0000","last_active":"2024-05-08T21:53:24.566197+0000","last_peered":"2024-05-08T21:53:24.566197+0000","last_clean":"2024-05-08T21:53:24.566197+0000","last_became_active":"2024-05-08T21:53:23.580154+0000","last_became_peered":"2024-05-08T21:53:23.580154+0000","last_unstale":"2024-05-08T21:53:24.566197+0000","last_undegraded":"2024-05-08T21:53:24.566197+0000","last_fullsized":"2024-05-08T21:53:24.566197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:50.713758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568936+0000","last_change":"2024-05-08T21:53:23.580549+0000","last_active":"2024-05-08T21:53:24.568936+0000","last_peered":"2024-05-08T21:53:24.568936+0000","last_clean":"2024-05-08T21:53:24.568936+0000","last_became_active":"2024-05-08T21:53:23.579962+0000","last_became_peered":"2024-05-08T21:53:23.579962+0000","last_unstale":"2024-05-08T21:53:24.568936+0000","last_undegraded":"2024-05-08T21:53:24.568936+0000","last_fullsized":"2024-05-08T21:53:24.568936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:52.010295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113920+0000","last_change":"2024-05-08T21:53:23.589640+0000","last_active":"2024-05-08T21:53:25.113920+0000","last_peered":"2024-05-08T21:53:25.113920+0000","last_clean":"2024-05-08T21:53:25.113920+0000","last_became_active":"2024-05-08T21:53:23.589347+0000","last_became_peered":"2024-05-08T21:53:23.589347+0000","last_unstale":"2024-05-08T21:53:25.113920+0000","last_undegraded":"2024-05-08T21:53:25.113920+0000","last_fullsized":"2024-05-08T21:53:25.113920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:40.346414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889087+0000","last_change":"2024-05-08T21:53:23.586430+0000","last_active":"2024-05-08T21:53:25.889087+0000","last_peered":"2024-05-08T21:53:25.889087+0000","last_clean":"2024-05-08T21:53:25.889087+0000","last_became_active":"2024-05-08T21:53:23.585986+0000","last_became_peered":"2024-05-08T21:53:23.585986+0000","last_unstale":"2024-05-08T21:53:25.889087+0000","last_undegraded":"2024-05-08T21:53:25.889087+0000","last_fullsized":"2024-05-08T21:53:25.889087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:09.970481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.586950+0000","last_change":"2024-05-08T21:53:23.585441+0000","last_active":"2024-05-08T21:53:23.586950+0000","last_peered":"2024-05-08T21:53:23.586950+0000","last_clean":"2024-05-08T21:53:23.586950+0000","last_became_active":"2024-05-08T21:53:23.584809+0000","last_became_peered":"2024-05-08T21:53:23.584809+0000","last_unstale":"2024-05-08T21:53:23.586950+0000","last_undegraded":"2024-05-08T21:53:23.586950+0000","last_fullsized":"2024-05-08T21:53:23.586950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:31.599185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806575+0000","last_change":"2024-05-08T21:53:23.575757+0000","last_active":"2024-05-08T21:53:23.806575+0000","last_peered":"2024-05-08T21:53:23.806575+0000","last_clean":"2024-05-08T21:53:23.806575+0000","last_became_active":"2024-05-08T21:53:23.575376+0000","last_became_peered":"2024-05-08T21:53:23.575376+0000","last_unstale":"2024-05-08T21:53:23.806575+0000","last_undegraded":"2024-05-08T21:53:23.806575+0000","last_fullsized":"2024-05-08T21:53:23.806575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.851616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779926+0000","last_change":"2024-05-08T21:53:23.573225+0000","last_active":"2024-05-08T21:53:23.779926+0000","last_peered":"2024-05-08T21:53:23.779926+0000","last_clean":"2024-05-08T21:53:23.779926+0000","last_became_active":"2024-05-08T21:53:23.572784+0000","last_became_peered":"2024-05-08T21:53:23.572784+0000","last_unstale":"2024-05-08T21:53:23.779926+0000","last_undegraded":"2024-05-08T21:53:23.779926+0000","last_fullsized":"2024-05-08T21:53:23.779926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:11.963959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779898+0000","last_change":"2024-05-08T21:53:23.583961+0000","last_active":"2024-05-08T21:53:23.779898+0000","last_peered":"2024-05-08T21:53:23.779898+0000","last_clean":"2024-05-08T21:53:23.779898+0000","last_became_active":"2024-05-08T21:53:23.583532+0000","last_became_peered":"2024-05-08T21:53:23.583532+0000","last_unstale":"2024-05-08T21:53:23.779898+0000","last_undegraded":"2024-05-08T21:53:23.779898+0000","last_fullsized":"2024-05-08T21:53:23.779898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:15:04.318298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568388+0000","last_change":"2024-05-08T21:53:23.577361+0000","last_active":"2024-05-08T21:53:24.568388+0000","last_peered":"2024-05-08T21:53:24.568388+0000","last_clean":"2024-05-08T21:53:24.568388+0000","last_became_active":"2024-05-08T21:53:23.577003+0000","last_became_peered":"2024-05-08T21:53:23.577003+0000","last_unstale":"2024-05-08T21:53:24.568388+0000","last_undegraded":"2024-05-08T21:53:24.568388+0000","last_fullsized":"2024-05-08T21:53:24.568388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:14.617211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703171+0000","last_change":"2024-05-08T21:53:23.579881+0000","last_active":"2024-05-08T21:53:23.703171+0000","last_peered":"2024-05-08T21:53:23.703171+0000","last_clean":"2024-05-08T21:53:23.703171+0000","last_became_active":"2024-05-08T21:53:23.579569+0000","last_became_peered":"2024-05-08T21:53:23.579569+0000","last_unstale":"2024-05-08T21:53:23.703171+0000","last_undegraded":"2024-05-08T21:53:23.703171+0000","last_fullsized":"2024-05-08T21:53:23.703171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:38.210018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.591923+0000","last_change":"2024-05-08T21:53:23.573027+0000","last_active":"2024-05-08T21:53:26.591923+0000","last_peered":"2024-05-08T21:53:26.591923+0000","last_clean":"2024-05-08T21:53:26.591923+0000","last_became_active":"2024-05-08T21:53:23.572709+0000","last_became_peered":"2024-05-08T21:53:23.572709+0000","last_unstale":"2024-05-08T21:53:26.591923+0000","last_undegraded":"2024-05-08T21:53:26.591923+0000","last_fullsized":"2024-05-08T21:53:26.591923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:36.555513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.588568+0000","last_change":"2024-05-08T21:53:23.586364+0000","last_active":"2024-05-08T21:53:23.588568+0000","last_peered":"2024-05-08T21:53:23.588568+0000","last_clean":"2024-05-08T21:53:23.588568+0000","last_became_active":"2024-05-08T21:53:23.585420+0000","last_became_peered":"2024-05-08T21:53:23.585420+0000","last_unstale":"2024-05-08T21:53:23.588568+0000","last_undegraded":"2024-05-08T21:53:23.588568+0000","last_fullsized":"2024-05-08T21:53:23.588568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:01.545821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.578302+0000","last_change":"2024-05-08T21:53:23.148809+0000","last_active":"2024-05-08T21:53:23.578302+0000","last_peered":"2024-05-08T21:53:23.578302+0000","last_clean":"2024-05-08T21:53:23.578302+0000","last_became_active":"2024-05-08T21:53:23.148532+0000","last_became_peered":"2024-05-08T21:53:23.148532+0000","last_unstale":"2024-05-08T21:53:23.578302+0000","last_undegraded":"2024-05-08T21:53:23.578302+0000","last_fullsized":"2024-05-08T21:53:23.578302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:48.209778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566739+0000","last_change":"2024-05-08T21:53:23.584120+0000","last_active":"2024-05-08T21:53:24.566739+0000","last_peered":"2024-05-08T21:53:24.566739+0000","last_clean":"2024-05-08T21:53:24.566739+0000","last_became_active":"2024-05-08T21:53:23.583804+0000","last_became_peered":"2024-05-08T21:53:23.583804+0000","last_unstale":"2024-05-08T21:53:24.566739+0000","last_undegraded":"2024-05-08T21:53:24.566739+0000","last_fullsized":"2024-05-08T21:53:24.566739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:10.669322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.196151+0000","last_change":"2024-05-08T21:53:23.144732+0000","last_active":"2024-05-08T21:53:26.196151+0000","last_peered":"2024-05-08T21:53:26.196151+0000","last_clean":"2024-05-08T21:53:26.196151+0000","last_became_active":"2024-05-08T21:53:23.144405+0000","last_became_peered":"2024-05-08T21:53:23.144405+0000","last_unstale":"2024-05-08T21:53:26.196151+0000","last_undegraded":"2024-05-08T21:53:26.196151+0000","last_fullsized":"2024-05-08T21:53:26.196151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:46.356499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780046+0000","last_change":"2024-05-08T21:53:23.578621+0000","last_active":"2024-05-08T21:53:23.780046+0000","last_peered":"2024-05-08T21:53:23.780046+0000","last_clean":"2024-05-08T21:53:23.780046+0000","last_became_active":"2024-05-08T21:53:23.578331+0000","last_became_peered":"2024-05-08T21:53:23.578331+0000","last_unstale":"2024-05-08T21:53:23.780046+0000","last_undegraded":"2024-05-08T21:53:23.780046+0000","last_fullsized":"2024-05-08T21:53:23.780046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:27.409751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954956+0000","last_change":"2024-05-08T21:53:23.145398+0000","last_active":"2024-05-08T21:53:27.954956+0000","last_peered":"2024-05-08T21:53:27.954956+0000","last_clean":"2024-05-08T21:53:27.954956+0000","last_became_active":"2024-05-08T21:53:23.144997+0000","last_became_peered":"2024-05-08T21:53:23.144997+0000","last_unstale":"2024-05-08T21:53:27.954956+0000","last_undegraded":"2024-05-08T21:53:27.954956+0000","last_fullsized":"2024-05-08T21:53:27.954956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.116791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568566+0000","last_change":"2024-05-08T21:53:23.573118+0000","last_active":"2024-05-08T21:53:24.568566+0000","last_peered":"2024-05-08T21:53:24.568566+0000","last_clean":"2024-05-08T21:53:24.568566+0000","last_became_active":"2024-05-08T21:53:23.572639+0000","last_became_peered":"2024-05-08T21:53:23.572639+0000","last_unstale":"2024-05-08T21:53:24.568566+0000","last_undegraded":"2024-05-08T21:53:24.568566+0000","last_fullsized":"2024-05-08T21:53:24.568566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.786457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.586989+0000","last_change":"2024-05-08T21:53:23.144706+0000","last_active":"2024-05-08T21:53:23.586989+0000","last_peered":"2024-05-08T21:53:23.586989+0000","last_clean":"2024-05-08T21:53:23.586989+0000","last_became_active":"2024-05-08T21:53:23.144380+0000","last_became_peered":"2024-05-08T21:53:23.144380+0000","last_unstale":"2024-05-08T21:53:23.586989+0000","last_undegraded":"2024-05-08T21:53:23.586989+0000","last_fullsized":"2024-05-08T21:53:23.586989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:05.594755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832631+0000","last_change":"2024-05-08T21:53:23.573836+0000","last_active":"2024-05-08T21:53:24.832631+0000","last_peered":"2024-05-08T21:53:24.832631+0000","last_clean":"2024-05-08T21:53:24.832631+0000","last_became_active":"2024-05-08T21:53:23.572323+0000","last_became_peered":"2024-05-08T21:53:23.572323+0000","last_unstale":"2024-05-08T21:53:24.832631+0000","last_undegraded":"2024-05-08T21:53:24.832631+0000","last_fullsized":"2024-05-08T21:53:24.832631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:34.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.948089+0000","last_change":"2024-05-08T21:53:23.147919+0000","last_active":"2024-05-08T21:53:27.948089+0000","last_peered":"2024-05-08T21:53:27.948089+0000","last_clean":"2024-05-08T21:53:27.948089+0000","last_became_active":"2024-05-08T21:53:23.147473+0000","last_became_peered":"2024-05-08T21:53:23.147473+0000","last_unstale":"2024-05-08T21:53:27.948089+0000","last_undegraded":"2024-05-08T21:53:27.948089+0000","last_fullsized":"2024-05-08T21:53:27.948089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:07.158398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566928+0000","last_change":"2024-05-08T21:53:23.576219+0000","last_active":"2024-05-08T21:53:24.566928+0000","last_peered":"2024-05-08T21:53:24.566928+0000","last_clean":"2024-05-08T21:53:24.566928+0000","last_became_active":"2024-05-08T21:53:23.575779+0000","last_became_peered":"2024-05-08T21:53:23.575779+0000","last_unstale":"2024-05-08T21:53:24.566928+0000","last_undegraded":"2024-05-08T21:53:24.566928+0000","last_fullsized":"2024-05-08T21:53:24.566928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:19.813711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953935+0000","last_change":"2024-05-08T21:53:23.152159+0000","last_active":"2024-05-08T21:53:27.953935+0000","last_peered":"2024-05-08T21:53:27.953935+0000","last_clean":"2024-05-08T21:53:27.953935+0000","last_became_active":"2024-05-08T21:53:23.151816+0000","last_became_peered":"2024-05-08T21:53:23.151816+0000","last_unstale":"2024-05-08T21:53:27.953935+0000","last_undegraded":"2024-05-08T21:53:27.953935+0000","last_fullsized":"2024-05-08T21:53:27.953935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.995097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832701+0000","last_change":"2024-05-08T21:53:23.575509+0000","last_active":"2024-05-08T21:53:24.832701+0000","last_peered":"2024-05-08T21:53:24.832701+0000","last_clean":"2024-05-08T21:53:24.832701+0000","last_became_active":"2024-05-08T21:53:23.574966+0000","last_became_peered":"2024-05-08T21:53:23.574966+0000","last_unstale":"2024-05-08T21:53:24.832701+0000","last_undegraded":"2024-05-08T21:53:24.832701+0000","last_fullsized":"2024-05-08T21:53:24.832701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:50.075249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.600882+0000","last_change":"2024-05-08T21:53:23.143151+0000","last_active":"2024-05-08T21:53:27.600882+0000","last_peered":"2024-05-08T21:53:27.600882+0000","last_clean":"2024-05-08T21:53:27.600882+0000","last_became_active":"2024-05-08T21:53:23.142829+0000","last_became_peered":"2024-05-08T21:53:23.142829+0000","last_unstale":"2024-05-08T21:53:27.600882+0000","last_undegraded":"2024-05-08T21:53:27.600882+0000","last_fullsized":"2024-05-08T21:53:27.600882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:49.372293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566098+0000","last_change":"2024-05-08T21:53:23.573587+0000","last_active":"2024-05-08T21:53:24.566098+0000","last_peered":"2024-05-08T21:53:24.566098+0000","last_clean":"2024-05-08T21:53:24.566098+0000","last_became_active":"2024-05-08T21:53:23.573329+0000","last_became_peered":"2024-05-08T21:53:23.573329+0000","last_unstale":"2024-05-08T21:53:24.566098+0000","last_undegraded":"2024-05-08T21:53:24.566098+0000","last_fullsized":"2024-05-08T21:53:24.566098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:49.915501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953090+0000","last_change":"2024-05-08T21:53:23.144147+0000","last_active":"2024-05-08T21:53:27.953090+0000","last_peered":"2024-05-08T21:53:27.953090+0000","last_clean":"2024-05-08T21:53:27.953090+0000","last_became_active":"2024-05-08T21:53:23.143808+0000","last_became_peered":"2024-05-08T21:53:23.143808+0000","last_unstale":"2024-05-08T21:53:27.953090+0000","last_undegraded":"2024-05-08T21:53:27.953090+0000","last_fullsized":"2024-05-08T21:53:27.953090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:07.011887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.592090+0000","last_change":"2024-05-08T21:53:23.583193+0000","last_active":"2024-05-08T21:53:26.592090+0000","last_peered":"2024-05-08T21:53:26.592090+0000","last_clean":"2024-05-08T21:53:26.592090+0000","last_became_active":"2024-05-08T21:53:23.582833+0000","last_became_peered":"2024-05-08T21:53:23.582833+0000","last_unstale":"2024-05-08T21:53:26.592090+0000","last_undegraded":"2024-05-08T21:53:26.592090+0000","last_fullsized":"2024-05-08T21:53:26.592090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:08.708225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.950589+0000","last_change":"2024-05-08T21:53:23.146398+0000","last_active":"2024-05-08T21:53:29.950589+0000","last_peered":"2024-05-08T21:53:29.950589+0000","last_clean":"2024-05-08T21:53:29.950589+0000","last_became_active":"2024-05-08T21:53:23.145823+0000","last_became_peered":"2024-05-08T21:53:23.145823+0000","last_unstale":"2024-05-08T21:53:29.950589+0000","last_undegraded":"2024-05-08T21:53:29.950589+0000","last_fullsized":"2024-05-08T21:53:29.950589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.535292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568901+0000","last_change":"2024-05-08T21:53:23.578664+0000","last_active":"2024-05-08T21:53:24.568901+0000","last_peered":"2024-05-08T21:53:24.568901+0000","last_clean":"2024-05-08T21:53:24.568901+0000","last_became_active":"2024-05-08T21:53:23.578398+0000","last_became_peered":"2024-05-08T21:53:23.578398+0000","last_unstale":"2024-05-08T21:53:24.568901+0000","last_undegraded":"2024-05-08T21:53:24.568901+0000","last_fullsized":"2024-05-08T21:53:24.568901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:10.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.961453+0000","last_change":"2024-05-08T21:53:23.151071+0000","last_active":"2024-05-08T21:53:28.961453+0000","last_peered":"2024-05-08T21:53:28.961453+0000","last_clean":"2024-05-08T21:53:28.961453+0000","last_became_active":"2024-05-08T21:53:23.150566+0000","last_became_peered":"2024-05-08T21:53:23.150566+0000","last_unstale":"2024-05-08T21:53:28.961453+0000","last_undegraded":"2024-05-08T21:53:28.961453+0000","last_fullsized":"2024-05-08T21:53:28.961453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:51.845419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780026+0000","last_change":"2024-05-08T21:53:23.573421+0000","last_active":"2024-05-08T21:53:23.780026+0000","last_peered":"2024-05-08T21:53:23.780026+0000","last_clean":"2024-05-08T21:53:23.780026+0000","last_became_active":"2024-05-08T21:53:23.571240+0000","last_became_peered":"2024-05-08T21:53:23.571240+0000","last_unstale":"2024-05-08T21:53:23.780026+0000","last_undegraded":"2024-05-08T21:53:23.780026+0000","last_fullsized":"2024-05-08T21:53:23.780026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:51.695322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.947244+0000","last_change":"2024-05-08T21:53:23.144552+0000","last_active":"2024-05-08T21:53:27.947244+0000","last_peered":"2024-05-08T21:53:27.947244+0000","last_clean":"2024-05-08T21:53:27.947244+0000","last_became_active":"2024-05-08T21:53:23.144231+0000","last_became_peered":"2024-05-08T21:53:23.144231+0000","last_unstale":"2024-05-08T21:53:27.947244+0000","last_undegraded":"2024-05-08T21:53:27.947244+0000","last_fullsized":"2024-05-08T21:53:27.947244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:44.877035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568575+0000","last_change":"2024-05-08T21:53:23.572913+0000","last_active":"2024-05-08T21:53:24.568575+0000","last_peered":"2024-05-08T21:53:24.568575+0000","last_clean":"2024-05-08T21:53:24.568575+0000","last_became_active":"2024-05-08T21:53:23.572256+0000","last_became_peered":"2024-05-08T21:53:23.572256+0000","last_unstale":"2024-05-08T21:53:24.568575+0000","last_undegraded":"2024-05-08T21:53:24.568575+0000","last_fullsized":"2024-05-08T21:53:24.568575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:05.485571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'13","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.972309+0000","last_change":"2024-05-08T21:53:23.150978+0000","last_active":"2024-05-08T21:53:29.972309+0000","last_peered":"2024-05-08T21:53:29.972309+0000","last_clean":"2024-05-08T21:53:29.972309+0000","last_became_active":"2024-05-08T21:53:23.150637+0000","last_became_peered":"2024-05-08T21:53:23.150637+0000","last_unstale":"2024-05-08T21:53:29.972309+0000","last_undegraded":"2024-05-08T21:53:29.972309+0000","last_fullsized":"2024-05-08T21:53:29.972309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:40.569414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8166,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566921+0000","last_change":"2024-05-08T21:53:23.581145+0000","last_active":"2024-05-08T21:53:24.566921+0000","last_peered":"2024-05-08T21:53:24.566921+0000","last_clean":"2024-05-08T21:53:24.566921+0000","last_became_active":"2024-05-08T21:53:23.580727+0000","last_became_peered":"2024-05-08T21:53:23.580727+0000","last_unstale":"2024-05-08T21:53:24.566921+0000","last_undegraded":"2024-05-08T21:53:24.566921+0000","last_fullsized":"2024-05-08T21:53:24.566921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:58:21.737504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'7","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.944484+0000","last_change":"2024-05-08T21:53:23.146278+0000","last_active":"2024-05-08T21:53:27.944484+0000","last_peered":"2024-05-08T21:53:27.944484+0000","last_clean":"2024-05-08T21:53:27.944484+0000","last_became_active":"2024-05-08T21:53:23.145965+0000","last_became_peered":"2024-05-08T21:53:23.145965+0000","last_unstale":"2024-05-08T21:53:27.944484+0000","last_undegraded":"2024-05-08T21:53:27.944484+0000","last_fullsized":"2024-05-08T21:53:27.944484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:05.370847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806557+0000","last_change":"2024-05-08T21:53:23.576732+0000","last_active":"2024-05-08T21:53:23.806557+0000","last_peered":"2024-05-08T21:53:23.806557+0000","last_clean":"2024-05-08T21:53:23.806557+0000","last_became_active":"2024-05-08T21:53:23.576415+0000","last_became_peered":"2024-05-08T21:53:23.576415+0000","last_unstale":"2024-05-08T21:53:23.806557+0000","last_undegraded":"2024-05-08T21:53:23.806557+0000","last_fullsized":"2024-05-08T21:53:23.806557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:47.184028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953150+0000","last_change":"2024-05-08T21:53:23.148398+0000","last_active":"2024-05-08T21:53:27.953150+0000","last_peered":"2024-05-08T21:53:27.953150+0000","last_clean":"2024-05-08T21:53:27.953150+0000","last_became_active":"2024-05-08T21:53:23.148078+0000","last_became_peered":"2024-05-08T21:53:23.148078+0000","last_unstale":"2024-05-08T21:53:27.953150+0000","last_undegraded":"2024-05-08T21:53:27.953150+0000","last_fullsized":"2024-05-08T21:53:27.953150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:38.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889064+0000","last_change":"2024-05-08T21:53:23.581700+0000","last_active":"2024-05-08T21:53:25.889064+0000","last_peered":"2024-05-08T21:53:25.889064+0000","last_clean":"2024-05-08T21:53:25.889064+0000","last_became_active":"2024-05-08T21:53:23.581346+0000","last_became_peered":"2024-05-08T21:53:23.581346+0000","last_unstale":"2024-05-08T21:53:25.889064+0000","last_undegraded":"2024-05-08T21:53:25.889064+0000","last_fullsized":"2024-05-08T21:53:25.889064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:08.530505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953772+0000","last_change":"2024-05-08T21:53:23.148415+0000","last_active":"2024-05-08T21:53:27.953772+0000","last_peered":"2024-05-08T21:53:27.953772+0000","last_clean":"2024-05-08T21:53:27.953772+0000","last_became_active":"2024-05-08T21:53:23.148037+0000","last_became_peered":"2024-05-08T21:53:23.148037+0000","last_unstale":"2024-05-08T21:53:27.953772+0000","last_undegraded":"2024-05-08T21:53:27.953772+0000","last_fullsized":"2024-05-08T21:53:27.953772+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:17.367009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888950+0000","last_change":"2024-05-08T21:53:23.589596+0000","last_active":"2024-05-08T21:53:25.888950+0000","last_peered":"2024-05-08T21:53:25.888950+0000","last_clean":"2024-05-08T21:53:25.888950+0000","last_became_active":"2024-05-08T21:53:23.589309+0000","last_became_peered":"2024-05-08T21:53:23.589309+0000","last_unstale":"2024-05-08T21:53:25.888950+0000","last_undegraded":"2024-05-08T21:53:25.888950+0000","last_fullsized":"2024-05-08T21:53:25.888950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:41.814532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'16","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.945110+0000","last_change":"2024-05-08T21:53:23.145434+0000","last_active":"2024-05-08T21:53:27.945110+0000","last_peered":"2024-05-08T21:53:27.945110+0000","last_clean":"2024-05-08T21:53:27.945110+0000","last_became_active":"2024-05-08T21:53:23.144328+0000","last_became_peered":"2024-05-08T21:53:23.144328+0000","last_unstale":"2024-05-08T21:53:27.945110+0000","last_undegraded":"2024-05-08T21:53:27.945110+0000","last_fullsized":"2024-05-08T21:53:27.945110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:30.389806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832474+0000","last_change":"2024-05-08T21:53:23.572862+0000","last_active":"2024-05-08T21:53:24.832474+0000","last_peered":"2024-05-08T21:53:24.832474+0000","last_clean":"2024-05-08T21:53:24.832474+0000","last_became_active":"2024-05-08T21:53:23.572516+0000","last_became_peered":"2024-05-08T21:53:23.572516+0000","last_unstale":"2024-05-08T21:53:24.832474+0000","last_undegraded":"2024-05-08T21:53:24.832474+0000","last_fullsized":"2024-05-08T21:53:24.832474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:37.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.567174+0000","last_change":"2024-05-08T21:53:23.146282+0000","last_active":"2024-05-08T21:53:24.567174+0000","last_peered":"2024-05-08T21:53:24.567174+0000","last_clean":"2024-05-08T21:53:24.567174+0000","last_became_active":"2024-05-08T21:53:23.145974+0000","last_became_peered":"2024-05-08T21:53:23.145974+0000","last_unstale":"2024-05-08T21:53:24.567174+0000","last_undegraded":"2024-05-08T21:53:24.567174+0000","last_fullsized":"2024-05-08T21:53:24.567174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:26.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703149+0000","last_change":"2024-05-08T21:53:23.578164+0000","last_active":"2024-05-08T21:53:23.703149+0000","last_peered":"2024-05-08T21:53:23.703149+0000","last_clean":"2024-05-08T21:53:23.703149+0000","last_became_active":"2024-05-08T21:53:23.577832+0000","last_became_peered":"2024-05-08T21:53:23.577832+0000","last_unstale":"2024-05-08T21:53:23.703149+0000","last_undegraded":"2024-05-08T21:53:23.703149+0000","last_fullsized":"2024-05-08T21:53:23.703149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:18.700875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.569471+0000","last_change":"2024-05-08T21:53:23.151368+0000","last_active":"2024-05-08T21:53:23.569471+0000","last_peered":"2024-05-08T21:53:23.569471+0000","last_clean":"2024-05-08T21:53:23.569471+0000","last_became_active":"2024-05-08T21:53:23.151068+0000","last_became_peered":"2024-05-08T21:53:23.151068+0000","last_unstale":"2024-05-08T21:53:23.569471+0000","last_undegraded":"2024-05-08T21:53:23.569471+0000","last_fullsized":"2024-05-08T21:53:23.569471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:42.662149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113749+0000","last_change":"2024-05-08T21:53:23.581387+0000","last_active":"2024-05-08T21:53:25.113749+0000","last_peered":"2024-05-08T21:53:25.113749+0000","last_clean":"2024-05-08T21:53:25.113749+0000","last_became_active":"2024-05-08T21:53:23.579346+0000","last_became_peered":"2024-05-08T21:53:23.579346+0000","last_unstale":"2024-05-08T21:53:25.113749+0000","last_undegraded":"2024-05-08T21:53:25.113749+0000","last_fullsized":"2024-05-08T21:53:25.113749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:42.997082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.971734+0000","last_change":"2024-05-08T21:53:23.150218+0000","last_active":"2024-05-08T21:53:29.971734+0000","last_peered":"2024-05-08T21:53:29.971734+0000","last_clean":"2024-05-08T21:53:29.971734+0000","last_became_active":"2024-05-08T21:53:23.148582+0000","last_became_peered":"2024-05-08T21:53:23.148582+0000","last_unstale":"2024-05-08T21:53:29.971734+0000","last_undegraded":"2024-05-08T21:53:29.971734+0000","last_fullsized":"2024-05-08T21:53:29.971734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:32.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113790+0000","last_change":"2024-05-08T21:53:23.579971+0000","last_active":"2024-05-08T21:53:25.113790+0000","last_peered":"2024-05-08T21:53:25.113790+0000","last_clean":"2024-05-08T21:53:25.113790+0000","last_became_active":"2024-05-08T21:53:23.578267+0000","last_became_peered":"2024-05-08T21:53:23.578267+0000","last_unstale":"2024-05-08T21:53:25.113790+0000","last_undegraded":"2024-05-08T21:53:25.113790+0000","last_fullsized":"2024-05-08T21:53:25.113790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:20.650914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832545+0000","last_change":"2024-05-08T21:53:23.584126+0000","last_active":"2024-05-08T21:53:24.832545+0000","last_peered":"2024-05-08T21:53:24.832545+0000","last_clean":"2024-05-08T21:53:24.832545+0000","last_became_active":"2024-05-08T21:53:23.583801+0000","last_became_peered":"2024-05-08T21:53:23.583801+0000","last_unstale":"2024-05-08T21:53:24.832545+0000","last_undegraded":"2024-05-08T21:53:24.832545+0000","last_fullsized":"2024-05-08T21:53:24.832545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:19.487948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.601829+0000","last_change":"2024-05-08T21:53:23.145221+0000","last_active":"2024-05-08T21:53:27.601829+0000","last_peered":"2024-05-08T21:53:27.601829+0000","last_clean":"2024-05-08T21:53:27.601829+0000","last_became_active":"2024-05-08T21:53:23.144659+0000","last_became_peered":"2024-05-08T21:53:23.144659+0000","last_unstale":"2024-05-08T21:53:27.601829+0000","last_undegraded":"2024-05-08T21:53:27.601829+0000","last_fullsized":"2024-05-08T21:53:27.601829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:58.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566506+0000","last_change":"2024-05-08T21:53:23.581748+0000","last_active":"2024-05-08T21:53:24.566506+0000","last_peered":"2024-05-08T21:53:24.566506+0000","last_clean":"2024-05-08T21:53:24.566506+0000","last_became_active":"2024-05-08T21:53:23.581336+0000","last_became_peered":"2024-05-08T21:53:23.581336+0000","last_unstale":"2024-05-08T21:53:24.566506+0000","last_undegraded":"2024-05-08T21:53:24.566506+0000","last_fullsized":"2024-05-08T21:53:24.566506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:31.158473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954240+0000","last_change":"2024-05-08T21:53:23.147073+0000","last_active":"2024-05-08T21:53:27.954240+0000","last_peered":"2024-05-08T21:53:27.954240+0000","last_clean":"2024-05-08T21:53:27.954240+0000","last_became_active":"2024-05-08T21:53:23.146720+0000","last_became_peered":"2024-05-08T21:53:23.146720+0000","last_unstale":"2024-05-08T21:53:27.954240+0000","last_undegraded":"2024-05-08T21:53:27.954240+0000","last_fullsized":"2024-05-08T21:53:27.954240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:40.566772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":321,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.573447+0000","last_change":"2024-05-08T21:52:33.304674+0000","last_active":"2024-05-08T21:53:23.573447+0000","last_peered":"2024-05-08T21:53:23.573447+0000","last_clean":"2024-05-08T21:53:23.573447+0000","last_became_active":"2024-05-08T21:52:33.304193+0000","last_became_peered":"2024-05-08T21:52:33.304193+0000","last_unstale":"2024-05-08T21:53:23.573447+0000","last_undegraded":"2024-05-08T21:53:23.573447+0000","last_fullsized":"2024-05-08T21:53:23.573447+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_clean_scrub_stamp":"2024-05-08T21:50:08.558194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:21.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806496+0000","last_change":"2024-05-08T21:53:23.583793+0000","last_active":"2024-05-08T21:53:23.806496+0000","last_peered":"2024-05-08T21:53:23.806496+0000","last_clean":"2024-05-08T21:53:23.806496+0000","last_became_active":"2024-05-08T21:53:23.583464+0000","last_became_peered":"2024-05-08T21:53:23.583464+0000","last_unstale":"2024-05-08T21:53:23.806496+0000","last_undegraded":"2024-05-08T21:53:23.806496+0000","last_fullsized":"2024-05-08T21:53:23.806496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:02.106595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.600923+0000","last_change":"2024-05-08T21:53:23.150460+0000","last_active":"2024-05-08T21:53:27.600923+0000","last_peered":"2024-05-08T21:53:27.600923+0000","last_clean":"2024-05-08T21:53:27.600923+0000","last_became_active":"2024-05-08T21:53:23.150025+0000","last_became_peered":"2024-05-08T21:53:23.150025+0000","last_unstale":"2024-05-08T21:53:27.600923+0000","last_undegraded":"2024-05-08T21:53:27.600923+0000","last_fullsized":"2024-05-08T21:53:27.600923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:28:09.311683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954703+0000","last_change":"2024-05-08T21:53:23.143160+0000","last_active":"2024-05-08T21:53:27.954703+0000","last_peered":"2024-05-08T21:53:27.954703+0000","last_clean":"2024-05-08T21:53:27.954703+0000","last_became_active":"2024-05-08T21:53:23.142718+0000","last_became_peered":"2024-05-08T21:53:23.142718+0000","last_unstale":"2024-05-08T21:53:27.954703+0000","last_undegraded":"2024-05-08T21:53:27.954703+0000","last_fullsized":"2024-05-08T21:53:27.954703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:54.148427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806605+0000","last_change":"2024-05-08T21:53:23.587069+0000","last_active":"2024-05-08T21:53:23.806605+0000","last_peered":"2024-05-08T21:53:23.806605+0000","last_clean":"2024-05-08T21:53:23.806605+0000","last_became_active":"2024-05-08T21:53:23.586782+0000","last_became_peered":"2024-05-08T21:53:23.586782+0000","last_unstale":"2024-05-08T21:53:23.806605+0000","last_undegraded":"2024-05-08T21:53:23.806605+0000","last_fullsized":"2024-05-08T21:53:23.806605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:08.897700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.567170+0000","last_change":"2024-05-08T21:53:23.142641+0000","last_active":"2024-05-08T21:53:23.567170+0000","last_peered":"2024-05-08T21:53:23.567170+0000","last_clean":"2024-05-08T21:53:23.567170+0000","last_became_active":"2024-05-08T21:53:23.142294+0000","last_became_peered":"2024-05-08T21:53:23.142294+0000","last_unstale":"2024-05-08T21:53:23.567170+0000","last_undegraded":"2024-05-08T21:53:23.567170+0000","last_fullsized":"2024-05-08T21:53:23.567170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:11.687406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566396+0000","last_change":"2024-05-08T21:53:23.580729+0000","last_active":"2024-05-08T21:53:24.566396+0000","last_peered":"2024-05-08T21:53:24.566396+0000","last_clean":"2024-05-08T21:53:24.566396+0000","last_became_active":"2024-05-08T21:53:23.580159+0000","last_became_peered":"2024-05-08T21:53:23.580159+0000","last_unstale":"2024-05-08T21:53:24.566396+0000","last_undegraded":"2024-05-08T21:53:24.566396+0000","last_fullsized":"2024-05-08T21:53:24.566396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:43:33.336308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.597286+0000","last_change":"2024-05-08T21:53:23.144242+0000","last_active":"2024-05-08T21:53:26.597286+0000","last_peered":"2024-05-08T21:53:26.597286+0000","last_clean":"2024-05-08T21:53:26.597286+0000","last_became_active":"2024-05-08T21:53:23.143905+0000","last_became_peered":"2024-05-08T21:53:23.143905+0000","last_unstale":"2024-05-08T21:53:26.597286+0000","last_undegraded":"2024-05-08T21:53:26.597286+0000","last_fullsized":"2024-05-08T21:53:26.597286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:41.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779991+0000","last_change":"2024-05-08T21:53:23.576972+0000","last_active":"2024-05-08T21:53:23.779991+0000","last_peered":"2024-05-08T21:53:23.779991+0000","last_clean":"2024-05-08T21:53:23.779991+0000","last_became_active":"2024-05-08T21:53:23.576668+0000","last_became_peered":"2024-05-08T21:53:23.576668+0000","last_unstale":"2024-05-08T21:53:23.779991+0000","last_undegraded":"2024-05-08T21:53:23.779991+0000","last_fullsized":"2024-05-08T21:53:23.779991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:25.545108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.595186+0000","last_change":"2024-05-08T21:53:23.145610+0000","last_active":"2024-05-08T21:53:26.595186+0000","last_peered":"2024-05-08T21:53:26.595186+0000","last_clean":"2024-05-08T21:53:26.595186+0000","last_became_active":"2024-05-08T21:53:23.145075+0000","last_became_peered":"2024-05-08T21:53:23.145075+0000","last_unstale":"2024-05-08T21:53:26.595186+0000","last_undegraded":"2024-05-08T21:53:26.595186+0000","last_fullsized":"2024-05-08T21:53:26.595186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:58.351303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566650+0000","last_change":"2024-05-08T21:53:23.579335+0000","last_active":"2024-05-08T21:53:24.566650+0000","last_peered":"2024-05-08T21:53:24.566650+0000","last_clean":"2024-05-08T21:53:24.566650+0000","last_became_active":"2024-05-08T21:53:23.578956+0000","last_became_peered":"2024-05-08T21:53:23.578956+0000","last_unstale":"2024-05-08T21:53:24.566650+0000","last_undegraded":"2024-05-08T21:53:24.566650+0000","last_fullsized":"2024-05-08T21:53:24.566650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:34.872954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.964020+0000","last_change":"2024-05-08T21:53:23.144108+0000","last_active":"2024-05-08T21:53:28.964020+0000","last_peered":"2024-05-08T21:53:28.964020+0000","last_clean":"2024-05-08T21:53:28.964020+0000","last_became_active":"2024-05-08T21:53:23.143786+0000","last_became_peered":"2024-05-08T21:53:23.143786+0000","last_unstale":"2024-05-08T21:53:28.964020+0000","last_undegraded":"2024-05-08T21:53:28.964020+0000","last_fullsized":"2024-05-08T21:53:28.964020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:15.512879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703096+0000","last_change":"2024-05-08T21:53:23.587667+0000","last_active":"2024-05-08T21:53:23.703096+0000","last_peered":"2024-05-08T21:53:23.703096+0000","last_clean":"2024-05-08T21:53:23.703096+0000","last_became_active":"2024-05-08T21:53:23.586908+0000","last_became_peered":"2024-05-08T21:53:23.586908+0000","last_unstale":"2024-05-08T21:53:23.703096+0000","last_undegraded":"2024-05-08T21:53:23.703096+0000","last_fullsized":"2024-05-08T21:53:23.703096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:06.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.601100+0000","last_change":"2024-05-08T21:53:23.150602+0000","last_active":"2024-05-08T21:53:24.601100+0000","last_peered":"2024-05-08T21:53:24.601100+0000","last_clean":"2024-05-08T21:53:24.601100+0000","last_became_active":"2024-05-08T21:53:23.150256+0000","last_became_peered":"2024-05-08T21:53:23.150256+0000","last_unstale":"2024-05-08T21:53:24.601100+0000","last_undegraded":"2024-05-08T21:53:24.601100+0000","last_fullsized":"2024-05-08T21:53:24.601100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:22:05.088246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566829+0000","last_change":"2024-05-08T21:53:23.577768+0000","last_active":"2024-05-08T21:53:24.566829+0000","last_peered":"2024-05-08T21:53:24.566829+0000","last_clean":"2024-05-08T21:53:24.566829+0000","last_became_active":"2024-05-08T21:53:23.577388+0000","last_became_peered":"2024-05-08T21:53:23.577388+0000","last_unstale":"2024-05-08T21:53:24.566829+0000","last_undegraded":"2024-05-08T21:53:24.566829+0000","last_fullsized":"2024-05-08T21:53:24.566829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:24.707904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.593957+0000","last_change":"2024-05-08T21:53:23.145165+0000","last_active":"2024-05-08T21:53:26.593957+0000","last_peered":"2024-05-08T21:53:26.593957+0000","last_clean":"2024-05-08T21:53:26.593957+0000","last_became_active":"2024-05-08T21:53:23.144465+0000","last_became_peered":"2024-05-08T21:53:23.144465+0000","last_unstale":"2024-05-08T21:53:26.593957+0000","last_undegraded":"2024-05-08T21:53:26.593957+0000","last_fullsized":"2024-05-08T21:53:26.593957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:47.229905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889021+0000","last_change":"2024-05-08T21:53:23.583476+0000","last_active":"2024-05-08T21:53:25.889021+0000","last_peered":"2024-05-08T21:53:25.889021+0000","last_clean":"2024-05-08T21:53:25.889021+0000","last_became_active":"2024-05-08T21:53:23.582989+0000","last_became_peered":"2024-05-08T21:53:23.582989+0000","last_unstale":"2024-05-08T21:53:25.889021+0000","last_undegraded":"2024-05-08T21:53:25.889021+0000","last_fullsized":"2024-05-08T21:53:25.889021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:05.402755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953612+0000","last_change":"2024-05-08T21:53:23.147469+0000","last_active":"2024-05-08T21:53:27.953612+0000","last_peered":"2024-05-08T21:53:27.953612+0000","last_clean":"2024-05-08T21:53:27.953612+0000","last_became_active":"2024-05-08T21:53:23.147145+0000","last_became_peered":"2024-05-08T21:53:23.147145+0000","last_unstale":"2024-05-08T21:53:27.953612+0000","last_undegraded":"2024-05-08T21:53:27.953612+0000","last_fullsized":"2024-05-08T21:53:27.953612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:15.806118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889101+0000","last_change":"2024-05-08T21:53:23.583221+0000","last_active":"2024-05-08T21:53:25.889101+0000","last_peered":"2024-05-08T21:53:25.889101+0000","last_clean":"2024-05-08T21:53:25.889101+0000","last_became_active":"2024-05-08T21:53:23.582942+0000","last_became_peered":"2024-05-08T21:53:23.582942+0000","last_unstale":"2024-05-08T21:53:25.889101+0000","last_undegraded":"2024-05-08T21:53:25.889101+0000","last_fullsized":"2024-05-08T21:53:25.889101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:32.188611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953180+0000","last_change":"2024-05-08T21:53:23.143721+0000","last_active":"2024-05-08T21:53:27.953180+0000","last_peered":"2024-05-08T21:53:27.953180+0000","last_clean":"2024-05-08T21:53:27.953180+0000","last_became_active":"2024-05-08T21:53:23.143227+0000","last_became_peered":"2024-05-08T21:53:23.143227+0000","last_unstale":"2024-05-08T21:53:27.953180+0000","last_undegraded":"2024-05-08T21:53:27.953180+0000","last_fullsized":"2024-05-08T21:53:27.953180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:09.619865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889075+0000","last_change":"2024-05-08T21:53:23.585820+0000","last_active":"2024-05-08T21:53:25.889075+0000","last_peered":"2024-05-08T21:53:25.889075+0000","last_clean":"2024-05-08T21:53:25.889075+0000","last_became_active":"2024-05-08T21:53:23.585435+0000","last_became_peered":"2024-05-08T21:53:23.585435+0000","last_unstale":"2024-05-08T21:53:25.889075+0000","last_undegraded":"2024-05-08T21:53:25.889075+0000","last_fullsized":"2024-05-08T21:53:25.889075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:48.696651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.185531+0000","last_change":"2024-05-08T21:53:23.141992+0000","last_active":"2024-05-08T21:53:26.185531+0000","last_peered":"2024-05-08T21:53:26.185531+0000","last_clean":"2024-05-08T21:53:26.185531+0000","last_became_active":"2024-05-08T21:53:23.141670+0000","last_became_peered":"2024-05-08T21:53:23.141670+0000","last_unstale":"2024-05-08T21:53:26.185531+0000","last_undegraded":"2024-05-08T21:53:26.185531+0000","last_fullsized":"2024-05-08T21:53:26.185531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:53.191681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889139+0000","last_change":"2024-05-08T21:53:23.586649+0000","last_active":"2024-05-08T21:53:25.889139+0000","last_peered":"2024-05-08T21:53:25.889139+0000","last_clean":"2024-05-08T21:53:25.889139+0000","last_became_active":"2024-05-08T21:53:23.586069+0000","last_became_peered":"2024-05-08T21:53:23.586069+0000","last_unstale":"2024-05-08T21:53:25.889139+0000","last_undegraded":"2024-05-08T21:53:25.889139+0000","last_fullsized":"2024-05-08T21:53:25.889139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:59.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703112+0000","last_change":"2024-05-08T21:53:23.588268+0000","last_active":"2024-05-08T21:53:23.703112+0000","last_peered":"2024-05-08T21:53:23.703112+0000","last_clean":"2024-05-08T21:53:23.703112+0000","last_became_active":"2024-05-08T21:53:23.587858+0000","last_became_peered":"2024-05-08T21:53:23.587858+0000","last_unstale":"2024-05-08T21:53:23.703112+0000","last_undegraded":"2024-05-08T21:53:23.703112+0000","last_fullsized":"2024-05-08T21:53:23.703112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:47.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.589133+0000","last_change":"2024-05-08T21:53:23.587227+0000","last_active":"2024-05-08T21:53:23.589133+0000","last_peered":"2024-05-08T21:53:23.589133+0000","last_clean":"2024-05-08T21:53:23.589133+0000","last_became_active":"2024-05-08T21:53:23.586807+0000","last_became_peered":"2024-05-08T21:53:23.586807+0000","last_unstale":"2024-05-08T21:53:23.589133+0000","last_undegraded":"2024-05-08T21:53:23.589133+0000","last_fullsized":"2024-05-08T21:53:23.589133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:29.814057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585531+0000","last_change":"2024-05-08T21:53:23.583968+0000","last_active":"2024-05-08T21:53:23.585531+0000","last_peered":"2024-05-08T21:53:23.585531+0000","last_clean":"2024-05-08T21:53:23.585531+0000","last_became_active":"2024-05-08T21:53:23.583625+0000","last_became_peered":"2024-05-08T21:53:23.583625+0000","last_unstale":"2024-05-08T21:53:23.585531+0000","last_undegraded":"2024-05-08T21:53:23.585531+0000","last_fullsized":"2024-05-08T21:53:23.585531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:40.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779949+0000","last_change":"2024-05-08T21:53:23.571284+0000","last_active":"2024-05-08T21:53:23.779949+0000","last_peered":"2024-05-08T21:53:23.779949+0000","last_clean":"2024-05-08T21:53:23.779949+0000","last_became_active":"2024-05-08T21:53:23.570915+0000","last_became_peered":"2024-05-08T21:53:23.570915+0000","last_unstale":"2024-05-08T21:53:23.779949+0000","last_undegraded":"2024-05-08T21:53:23.779949+0000","last_fullsized":"2024-05-08T21:53:23.779949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:52.562178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806616+0000","last_change":"2024-05-08T21:53:23.572011+0000","last_active":"2024-05-08T21:53:23.806616+0000","last_peered":"2024-05-08T21:53:23.806616+0000","last_clean":"2024-05-08T21:53:23.806616+0000","last_became_active":"2024-05-08T21:53:23.571788+0000","last_became_peered":"2024-05-08T21:53:23.571788+0000","last_unstale":"2024-05-08T21:53:23.806616+0000","last_undegraded":"2024-05-08T21:53:23.806616+0000","last_fullsized":"2024-05-08T21:53:23.806616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:43.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703137+0000","last_change":"2024-05-08T21:53:23.582823+0000","last_active":"2024-05-08T21:53:23.703137+0000","last_peered":"2024-05-08T21:53:23.703137+0000","last_clean":"2024-05-08T21:53:23.703137+0000","last_became_active":"2024-05-08T21:53:23.582366+0000","last_became_peered":"2024-05-08T21:53:23.582366+0000","last_unstale":"2024-05-08T21:53:23.703137+0000","last_undegraded":"2024-05-08T21:53:23.703137+0000","last_fullsized":"2024-05-08T21:53:23.703137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:44.029915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585670+0000","last_change":"2024-05-08T21:53:23.583566+0000","last_active":"2024-05-08T21:53:23.585670+0000","last_peered":"2024-05-08T21:53:23.585670+0000","last_clean":"2024-05-08T21:53:23.585670+0000","last_became_active":"2024-05-08T21:53:23.583204+0000","last_became_peered":"2024-05-08T21:53:23.583204+0000","last_unstale":"2024-05-08T21:53:23.585670+0000","last_undegraded":"2024-05-08T21:53:23.585670+0000","last_fullsized":"2024-05-08T21:53:23.585670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:41.426005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566280+0000","last_change":"2024-05-08T21:53:23.587105+0000","last_active":"2024-05-08T21:53:24.566280+0000","last_peered":"2024-05-08T21:53:24.566280+0000","last_clean":"2024-05-08T21:53:24.566280+0000","last_became_active":"2024-05-08T21:53:23.586773+0000","last_became_peered":"2024-05-08T21:53:23.586773+0000","last_unstale":"2024-05-08T21:53:24.566280+0000","last_undegraded":"2024-05-08T21:53:24.566280+0000","last_fullsized":"2024-05-08T21:53:24.566280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:50.809186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832696+0000","last_change":"2024-05-08T21:53:23.576779+0000","last_active":"2024-05-08T21:53:24.832696+0000","last_peered":"2024-05-08T21:53:24.832696+0000","last_clean":"2024-05-08T21:53:24.832696+0000","last_became_active":"2024-05-08T21:53:23.576434+0000","last_became_peered":"2024-05-08T21:53:23.576434+0000","last_unstale":"2024-05-08T21:53:24.832696+0000","last_undegraded":"2024-05-08T21:53:24.832696+0000","last_fullsized":"2024-05-08T21:53:24.832696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:10.168522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566243+0000","last_change":"2024-05-08T21:53:23.585196+0000","last_active":"2024-05-08T21:53:24.566243+0000","last_peered":"2024-05-08T21:53:24.566243+0000","last_clean":"2024-05-08T21:53:24.566243+0000","last_became_active":"2024-05-08T21:53:23.584160+0000","last_became_peered":"2024-05-08T21:53:23.584160+0000","last_unstale":"2024-05-08T21:53:24.566243+0000","last_undegraded":"2024-05-08T21:53:24.566243+0000","last_fullsized":"2024-05-08T21:53:24.566243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:15.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568545+0000","last_change":"2024-05-08T21:53:23.577887+0000","last_active":"2024-05-08T21:53:24.568545+0000","last_peered":"2024-05-08T21:53:24.568545+0000","last_clean":"2024-05-08T21:53:24.568545+0000","last_became_active":"2024-05-08T21:53:23.577597+0000","last_became_peered":"2024-05-08T21:53:23.577597+0000","last_unstale":"2024-05-08T21:53:24.568545+0000","last_undegraded":"2024-05-08T21:53:24.568545+0000","last_fullsized":"2024-05-08T21:53:24.568545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:33.704002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806594+0000","last_change":"2024-05-08T21:53:23.575180+0000","last_active":"2024-05-08T21:53:23.806594+0000","last_peered":"2024-05-08T21:53:23.806594+0000","last_clean":"2024-05-08T21:53:23.806594+0000","last_became_active":"2024-05-08T21:53:23.574521+0000","last_became_peered":"2024-05-08T21:53:23.574521+0000","last_unstale":"2024-05-08T21:53:23.806594+0000","last_undegraded":"2024-05-08T21:53:23.806594+0000","last_fullsized":"2024-05-08T21:53:23.806594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:59.609492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703125+0000","last_change":"2024-05-08T21:53:23.585380+0000","last_active":"2024-05-08T21:53:23.703125+0000","last_peered":"2024-05-08T21:53:23.703125+0000","last_clean":"2024-05-08T21:53:23.703125+0000","last_became_active":"2024-05-08T21:53:23.585073+0000","last_became_peered":"2024-05-08T21:53:23.585073+0000","last_unstale":"2024-05-08T21:53:23.703125+0000","last_undegraded":"2024-05-08T21:53:23.703125+0000","last_fullsized":"2024-05-08T21:53:23.703125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:33.572410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13971,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":36,"num_read_kb":29,"num_write":161,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":58587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":860160,"data_stored":410274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":10,"up_from":67,"seq":287762808847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1495040,"data_stored":1003205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.44}]},{"osd":3,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1,"5min":1,"15min":1},"last":0.502}]},{"osd":4,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.72799999999999998}]},{"osd":9,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.73999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972369,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":401142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.377}]},{"osd":1,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.65200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":8,"up_from":55,"seq":236223201301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":856064,"data_stored":409425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005}]},{"osd":3,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.83199999999999996}]},{"osd":5,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.624},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.85499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1474560,"data_stored":1003675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33800000000000002}]},{"osd":2,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.498}]},{"osd":4,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.76800000000000002}]},{"osd":6,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47899999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.749}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58699999999999997}]},{"osd":10,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":409242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.28899999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.432},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.437},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.871,"15min":0.871},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed May 8 21:52:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.748},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71899999999999997}]},{"osd":8,"last update":"Wed May 8 21:52:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70599999999999996}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.5},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.78500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed May 8 21:53:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.749,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.78400000000000003}]},{"osd":3,"last update":"Wed May 8 21:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed May 8 21:52:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64700000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.33700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.748},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.83999999999999997}]},{"osd":6,"last update":"Wed May 8 21:53:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.80200000000000005}]},{"osd":8,"last update":"Wed May 8 21:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.94299999999999995}]},{"osd":9,"last update":"Wed May 8 21:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.85399999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.92100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":411148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.498,"15min":0.46899999999999997},"min":{"1min":0.44600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48799999999999999,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.496}]},{"osd":2,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.83599999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed May 8 21:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed May 8 21:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Wed May 8 21:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed May 8 21:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73199999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":411996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48999999999999999,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed May 8 21:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82099999999999995}]},{"osd":5,"last update":"Wed May 8 21:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed May 8 21:53:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71999999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82799999999999996,"5min":0.879,"15min":0.879},"last":0.75700000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52900000000000003}]},{"osd":8,"last update":"Wed May 8 21:52:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.80300000000000005}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]}]},{"osd":3,"up_from":24,"seq":103079215144,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":409677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.34699999999999998},"min":{"1min":0.28999999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 8 21:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.629},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.375,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76500000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed May 8 21:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.502}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed May 8 21:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.627},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.48599999999999999}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.877}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":409748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.377,"15min":0.377},"max":{"1min":0.76800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.755}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.77300000000000002}]},{"osd":7,"last update":"Wed May 8 21:52:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.875},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.84499999999999997}]},{"osd":8,"last update":"Wed May 8 21:52:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91400000000000003}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.0109999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1482752,"data_stored":1001939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.77000000000000002}]},{"osd":1,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.37,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.70399999999999996}]},{"osd":2,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.253,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.441},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48699999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.78100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59899999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84499999999999997}]},{"osd":6,"last update":"Wed May 8 21:53:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55900000000000005}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.78500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":32107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":10247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:53:33.226 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T21:53:33.226 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T21:53:33.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T21:53:33.294 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 21:53:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: 2024-05-08T21:53:33.063+0000 7f965138e700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-08T21:53:33.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:32 smithi160 ceph-mon[46962]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:33.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:32 smithi160 ceph-mon[46962]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:33.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:32 smithi160 ceph-mon[46962]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T21:53:33.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:32 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/779800366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:33.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:32 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2837927255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:33.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:32 smithi186 ceph-mon[46619]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:33.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:32 smithi186 ceph-mon[46619]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:53:33.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:32 smithi186 ceph-mon[46619]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T21:53:33.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:32 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/779800366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:53:33.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:32 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2837927255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:33.484 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:34.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:33 smithi064 ceph-mon[42500]: pgmap v225: 97 pgs: 97 active+clean; 590 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7.0 KiB/s wr, 23 op/s 2024-05-08T21:53:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:33 smithi064 ceph-mon[42500]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:53:34.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:33 smithi160 ceph-mon[46962]: pgmap v225: 97 pgs: 97 active+clean; 590 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7.0 KiB/s wr, 23 op/s 2024-05-08T21:53:34.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:33 smithi160 ceph-mon[46962]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:53:34.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:33 smithi186 ceph-mon[46619]: pgmap v225: 97 pgs: 97 active+clean; 590 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7.0 KiB/s wr, 23 op/s 2024-05-08T21:53:34.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:33 smithi186 ceph-mon[46619]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:53:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.422+0000 7f05e0143700 1 -- 172.21.15.64:0/2920575782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8106cb0 msgr2=0x7f05d8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.422+0000 7f05e0143700 1 --2- 172.21.15.64:0/2920575782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8106cb0 0x7f05d8109120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05d000b0a0 tx=0x7f05d000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 -- 172.21.15.64:0/2920575782 shutdown_connections 2024-05-08T21:53:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 --2- 172.21.15.64:0/2920575782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05d8101a50 0x7f05d8103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 --2- 172.21.15.64:0/2920575782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8106cb0 0x7f05d8109120 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 --2- 172.21.15.64:0/2920575782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05d8104380 0x7f05d8106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 -- 172.21.15.64:0/2920575782 >> 172.21.15.64:0/2920575782 conn(0x7f05d80fb460 msgr2=0x7f05d80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 -- 172.21.15.64:0/2920575782 shutdown_connections 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 -- 172.21.15.64:0/2920575782 wait complete. 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.423+0000 7f05e0143700 1 Processor -- start 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 -- start start 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05d8101a50 0x7f05d8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 0x7f05d8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05d8106cb0 0x7f05d819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d819eb10 con 0x7f05d8104380 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d819ec50 con 0x7f05d8101a50 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05e0143700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d819ede0 con 0x7f05d8106cb0 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 0x7f05d8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 0x7f05d8197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59786/0 (socket says 172.21.15.64:59786) 2024-05-08T21:53:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 -- 172.21.15.64:0/948033680 learned_addr learned my addr 172.21.15.64:0/948033680 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05de6e0700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05d8106cb0 0x7f05d819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05ddedf700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05d8101a50 0x7f05d8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 -- 172.21.15.64:0/948033680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05d8106cb0 msgr2=0x7f05d819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05d8106cb0 0x7f05d819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 -- 172.21.15.64:0/948033680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05d8101a50 msgr2=0x7f05d8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05d8101a50 0x7f05d8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 -- 172.21.15.64:0/948033680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d0009d00 con 0x7f05d8104380 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.424+0000 7f05dd6de700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 0x7f05d8197510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05d400b560 tx=0x7f05d400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.425+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4011830 con 0x7f05d8104380 2024-05-08T21:53:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.425+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d819f0c0 con 0x7f05d8104380 2024-05-08T21:53:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.425+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05d4011e50 con 0x7f05d8104380 2024-05-08T21:53:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.425+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4010e50 con 0x7f05d8104380 2024-05-08T21:53:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.425+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d819f610 con 0x7f05d8104380 2024-05-08T21:53:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.426+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d80fcfe0 con 0x7f05d8104380 2024-05-08T21:53:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.427+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f05d40104f0 con 0x7f05d8104380 2024-05-08T21:53:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.427+0000 7f05ceffd700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f05c40a11b0 0x7f05c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.428+0000 7f05ddedf700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f05c40a11b0 0x7f05c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.428+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05d40cc6d0 con 0x7f05d8104380 2024-05-08T21:53:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.428+0000 7f05ddedf700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f05c40a11b0 0x7f05c40a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05c800ac50 tx=0x7f05c800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.431+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05d4095fc0 con 0x7f05d8104380 2024-05-08T21:53:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.722+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f05d810ca10 con 0x7f05d8104380 2024-05-08T21:53:34.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.727+0000 7f05ceffd700 1 -- 172.21.15.64:0/948033680 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f05d40995e0 con 0x7f05d8104380 2024-05-08T21:53:34.728 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-05-08T21:53:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBe9DtmltcxKxAAB6hZHf9U/xsNLWQJpUpPQA== 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f05c40a11b0 msgr2=0x7f05c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f05c40a11b0 0x7f05c40a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05c800ac50 tx=0x7f05c800a380 comp rx=0 tx=0).stop 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 msgr2=0x7f05d8197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 0x7f05d8197510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05d400b560 tx=0x7f05d400b870 comp rx=0 tx=0).stop 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 shutdown_connections 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f05c40a11b0 0x7f05c40a3670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05d8101a50 0x7f05d8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05d8106cb0 0x7f05d819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 --2- 172.21.15.64:0/948033680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8104380 0x7f05d8197510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 >> 172.21.15.64:0/948033680 conn(0x7f05d80fb460 msgr2=0x7f05d8102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 shutdown_connections 2024-05-08T21:53:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:34.729+0000 7f05e0143700 1 -- 172.21.15.64:0/948033680 wait complete. 2024-05-08T21:53:35.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:34 smithi064 ceph-mon[42500]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:53:35.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:34 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/948033680' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T21:53:35.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:34 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/948033680' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T21:53:35.105 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:53:35.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T21:53:35.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T21:53:35.148 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T21:53:35.148 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T21:53:35.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T21:53:35.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:34 smithi160 ceph-mon[46962]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:53:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:34 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/948033680' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T21:53:35.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:34 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/948033680' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T21:53:35.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:34 smithi186 ceph-mon[46619]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:53:35.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:34 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/948033680' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T21:53:35.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:34 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/948033680' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T21:53:35.416 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:35 smithi064 ceph-mon[42500]: pgmap v226: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.2 KiB/s wr, 26 op/s 2024-05-08T21:53:36.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:35 smithi160 ceph-mon[46962]: pgmap v226: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.2 KiB/s wr, 26 op/s 2024-05-08T21:53:36.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:35 smithi186 ceph-mon[46619]: pgmap v226: 97 pgs: 97 active+clean; 591 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.2 KiB/s wr, 26 op/s 2024-05-08T21:53:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 -- 172.21.15.64:0/183793959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb840ffed0 msgr2=0x7fbb84100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/183793959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb840ffed0 0x7fbb84100350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009990 tx=0x7fbb78009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 -- 172.21.15.64:0/183793959 shutdown_connections 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/183793959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb84100890 0x7fbb8410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/183793959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb84105f00 0x7fbb841062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/183793959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb840ffed0 0x7fbb84100350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 -- 172.21.15.64:0/183793959 >> 172.21.15.64:0/183793959 conn(0x7fbb84075f80 msgr2=0x7fbb84076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 -- 172.21.15.64:0/183793959 shutdown_connections 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 -- 172.21.15.64:0/183793959 wait complete. 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.455+0000 7fbb88b8b700 1 Processor -- start 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 -- start start 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb840ffed0 0x7fbb841981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:36.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb84100890 0x7fbb84198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 0x7fbb841a3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84198fe0 con 0x7fbb84105f00 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841a37a0 con 0x7fbb840ffed0 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb88b8b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841a38e0 con 0x7fbb84100890 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 0x7fbb841a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 0x7fbb841a3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59798/0 (socket says 172.21.15.64:59798) 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 -- 172.21.15.64:0/3881035161 learned_addr learned my addr 172.21.15.64:0/3881035161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb81d9b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb84100890 0x7fbb84198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb8259c700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb840ffed0 0x7fbb841981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 -- 172.21.15.64:0/3881035161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb84100890 msgr2=0x7fbb84198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb84100890 0x7fbb84198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 -- 172.21.15.64:0/3881035161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb840ffed0 msgr2=0x7fbb841981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:36.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb840ffed0 0x7fbb841981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 -- 172.21.15.64:0/3881035161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780095f0 con 0x7fbb84105f00 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.456+0000 7fbb82d9d700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 0x7fbb841a3260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c007ee0 tx=0x7fbb7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.457+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c017070 con 0x7fbb84105f00 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.457+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb841a3bc0 con 0x7fbb84105f00 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.457+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7c00f040 con 0x7fbb84105f00 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.457+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c005740 con 0x7fbb84105f00 2024-05-08T21:53:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.457+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841a40c0 con 0x7fbb84105f00 2024-05-08T21:53:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.458+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb84108420 con 0x7fbb84105f00 2024-05-08T21:53:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.459+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbb7c00dc60 con 0x7fbb84105f00 2024-05-08T21:53:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.459+0000 7fbb737fe700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb6c0a11b0 0x7fbb6c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.459+0000 7fbb8259c700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb6c0a11b0 0x7fbb6c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.459+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbb7c0cc710 con 0x7fbb84105f00 2024-05-08T21:53:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.460+0000 7fbb8259c700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb6c0a11b0 0x7fbb6c0a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb7400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.463+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbb7c095ef0 con 0x7fbb84105f00 2024-05-08T21:53:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.726+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbb841015b0 con 0x7fbb84105f00 2024-05-08T21:53:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.731+0000 7fbb737fe700 1 -- 172.21.15.64:0/3881035161 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260285 (secure 0 0 0) 0x7fbb7c099510 con 0x7fbb84105f00 2024-05-08T21:53:36.735 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:36.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb6c0a11b0 msgr2=0x7fbb6c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:36.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb6c0a11b0 0x7fbb6c0a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb7400af20 comp rx=0 tx=0).stop 2024-05-08T21:53:36.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 msgr2=0x7fbb841a3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:36.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 0x7fbb841a3260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c007ee0 tx=0x7fbb7c00d3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 shutdown_connections 2024-05-08T21:53:36.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbb6c0a11b0 0x7fbb6c0a3670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb840ffed0 0x7fbb841981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb84105f00 0x7fbb841a3260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 --2- 172.21.15.64:0/3881035161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb84100890 0x7fbb84198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 >> 172.21.15.64:0/3881035161 conn(0x7fbb84075f80 msgr2=0x7fbb840fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 shutdown_connections 2024-05-08T21:53:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:36.734+0000 7fbb88b8b700 1 -- 172.21.15.64:0/3881035161 wait complete. 2024-05-08T21:53:37.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:36 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3881035161' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T21:53:37.122 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3946872331},{"type":"v1","addr":"172.21.15.64:6801","nonce":3946872331}]},"active_addr":"172.21.15.64:6801/3946872331","active_change":"2024-05-08T21:48:20.034572+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1922858447}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":3517323888}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1958414101}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1577677719}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":696003545}]}]} 2024-05-08T21:53:37.126 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T21:53:37.126 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T21:53:37.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T21:53:37.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:36 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3881035161' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T21:53:37.358 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:37.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:36 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3881035161' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T21:53:38.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:37 smithi064 ceph-mon[42500]: pgmap v227: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.6 KiB/s wr, 27 op/s 2024-05-08T21:53:38.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:37 smithi160 ceph-mon[46962]: pgmap v227: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.6 KiB/s wr, 27 op/s 2024-05-08T21:53:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 -- 172.21.15.64:0/3719926782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 msgr2=0x7f1714105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 --2- 172.21.15.64:0/3719926782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f1714105bd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1704009990 tx=0x7f1704009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 -- 172.21.15.64:0/3719926782 shutdown_connections 2024-05-08T21:53:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 --2- 172.21.15.64:0/3719926782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1714101250 0x7f1714101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 --2- 172.21.15.64:0/3719926782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17141062b0 0x7f171410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 --2- 172.21.15.64:0/3719926782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f1714105bd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.362+0000 7f171b51c700 1 -- 172.21.15.64:0/3719926782 >> 172.21.15.64:0/3719926782 conn(0x7f1714078100 msgr2=0x7f1714078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.363+0000 7f171b51c700 1 -- 172.21.15.64:0/3719926782 shutdown_connections 2024-05-08T21:53:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.363+0000 7f171b51c700 1 -- 172.21.15.64:0/3719926782 wait complete. 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.363+0000 7f171b51c700 1 Processor -- start 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.363+0000 7f171b51c700 1 -- start start 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1714101250 0x7f17140732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f17140737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17141062b0 0x7f1714073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141a5790 con 0x7f1714101c00 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141a5900 con 0x7f17141062b0 2024-05-08T21:53:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141a5ac0 con 0x7f1714101250 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f17140737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f17140737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59812/0 (socket says 172.21.15.64:59812) 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 -- 172.21.15.64:0/3056683008 learned_addr learned my addr 172.21.15.64:0/3056683008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f17192b8700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1714101250 0x7f17140732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1719ab9700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17141062b0 0x7f1714073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 -- 172.21.15.64:0/3056683008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1714101250 msgr2=0x7f17140732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1714101250 0x7f17140732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 -- 172.21.15.64:0/3056683008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17141062b0 msgr2=0x7f1714073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17141062b0 0x7f1714073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 -- 172.21.15.64:0/3056683008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040095f0 con 0x7f1714101c00 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1719ab9700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17141062b0 0x7f1714073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f17192b8700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1714101250 0x7f17140732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f1718ab7700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f17140737e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17040098e0 tx=0x7f170400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170401d070 con 0x7f1714101c00 2024-05-08T21:53:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f170400fb90 con 0x7f1714101c00 2024-05-08T21:53:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17141a5d40 con 0x7f1714101c00 2024-05-08T21:53:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704017650 con 0x7f1714101c00 2024-05-08T21:53:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.364+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17141a61b0 con 0x7f1714101c00 2024-05-08T21:53:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.365+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17141095d0 con 0x7f1714101c00 2024-05-08T21:53:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.366+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f170400fd00 con 0x7f1714101c00 2024-05-08T21:53:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.367+0000 7f170a7fc700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f17000a1230 0x7f17000a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.367+0000 7f17192b8700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f17000a1230 0x7f17000a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.367+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f17040cdc30 con 0x7f1714101c00 2024-05-08T21:53:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.368+0000 7f17192b8700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f17000a1230 0x7f17000a36f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1710005d90 tx=0x7f1710005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.371+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f17040974a0 con 0x7f1714101c00 2024-05-08T21:53:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:37 smithi186 ceph-mon[46619]: pgmap v227: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.6 KiB/s wr, 27 op/s 2024-05-08T21:53:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.572+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17141026b0 con 0x7f1714101c00 2024-05-08T21:53:38.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.573+0000 7f170a7fc700 1 -- 172.21.15.64:0/3056683008 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f170409aac0 con 0x7f1714101c00 2024-05-08T21:53:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:38.574 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:24.557465+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f17000a1230 msgr2=0x7f17000a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f17000a1230 0x7f17000a36f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1710005d90 tx=0x7f1710005d00 comp rx=0 tx=0).stop 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 msgr2=0x7f17140737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f17140737e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17040098e0 tx=0x7f170400f720 comp rx=0 tx=0).stop 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 shutdown_connections 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f17000a1230 0x7f17000a36f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17141062b0 0x7f1714073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1714101250 0x7f17140732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 --2- 172.21.15.64:0/3056683008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1714101c00 0x7f17140737e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.575+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 >> 172.21.15.64:0/3056683008 conn(0x7f1714078100 msgr2=0x7f1714107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.576+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 shutdown_connections 2024-05-08T21:53:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:38.576+0000 7f171b51c700 1 -- 172.21.15.64:0/3056683008 wait complete. 2024-05-08T21:53:38.948 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T21:53:38.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T21:53:39.182 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:39.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:38 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3056683008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:39.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:38 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3056683008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:39.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:38 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3056683008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:40.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 -- 172.21.15.64:0/3899251411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64980ffba0 msgr2=0x7f64980fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:40.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 --2- 172.21.15.64:0/3899251411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64980ffba0 0x7f64980fff80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64880099c0 tx=0x7f6488009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 -- 172.21.15.64:0/3899251411 shutdown_connections 2024-05-08T21:53:40.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 --2- 172.21.15.64:0/3899251411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64981063c0 0x7f649810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 --2- 172.21.15.64:0/3899251411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64980ffba0 0x7f64980fff80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 --2- 172.21.15.64:0/3899251411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6498100550 0x7f6498105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 -- 172.21.15.64:0/3899251411 >> 172.21.15.64:0/3899251411 conn(0x7f6498075260 msgr2=0x7f6498075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 -- 172.21.15.64:0/3899251411 shutdown_connections 2024-05-08T21:53:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.171+0000 7f649e8e6700 1 -- 172.21.15.64:0/3899251411 wait complete. 2024-05-08T21:53:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 Processor -- start 2024-05-08T21:53:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 -- start start 2024-05-08T21:53:40.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64980ffba0 0x7f6498197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:40.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6498100550 0x7f64981977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:40.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64981063c0 0x7f649819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649819c190 con 0x7f64981063c0 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649819c300 con 0x7f6498100550 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649e8e6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649819c4c0 con 0x7f64980ffba0 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f64977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6498100550 0x7f64981977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f64977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6498100550 0x7f64981977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:42586/0 (socket says 172.21.15.64:42586) 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f64977fe700 1 -- 172.21.15.64:0/2993620338 learned_addr learned my addr 172.21.15.64:0/2993620338 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f649ce83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64981063c0 0x7f649819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.172+0000 7f6497fff700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64980ffba0 0x7f6498197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649ce83700 1 -- 172.21.15.64:0/2993620338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64980ffba0 msgr2=0x7f6498197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649ce83700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64980ffba0 0x7f6498197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649ce83700 1 -- 172.21.15.64:0/2993620338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6498100550 msgr2=0x7f64981977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649ce83700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6498100550 0x7f64981977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649ce83700 1 -- 172.21.15.64:0/2993620338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880095f0 con 0x7f64981063c0 2024-05-08T21:53:40.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649ce83700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64981063c0 0x7f649819bb80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6490007f10 tx=0x7f649000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:40.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490004db0 con 0x7f64981063c0 2024-05-08T21:53:40.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649819c7a0 con 0x7f64981063c0 2024-05-08T21:53:40.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f649000f040 con 0x7f64981063c0 2024-05-08T21:53:40.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490005990 con 0x7f64981063c0 2024-05-08T21:53:40.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.173+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64981a4000 con 0x7f64981063c0 2024-05-08T21:53:40.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.174+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64981095a0 con 0x7f64981063c0 2024-05-08T21:53:40.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.175+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f649000db20 con 0x7f64981063c0 2024-05-08T21:53:40.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.176+0000 7f64957fa700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64840a11b0 0x7f64840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:40.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.176+0000 7f6497fff700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64840a11b0 0x7f64840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:40.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.176+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f64900cc9a0 con 0x7f64981063c0 2024-05-08T21:53:40.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.177+0000 7f6497fff700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64840a11b0 0x7f64840a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6488009f50 tx=0x7f6488005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:40.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.179+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6490096180 con 0x7f64981063c0 2024-05-08T21:53:40.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:40 smithi160 ceph-mon[46962]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.9 KiB/s wr, 26 op/s 2024-05-08T21:53:40.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.383+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64981985a0 con 0x7f64981063c0 2024-05-08T21:53:40.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.384+0000 7f64957fa700 1 -- 172.21.15.64:0/2993620338 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f6490023020 con 0x7f64981063c0 2024-05-08T21:53:40.385 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:53:40.385 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:24.557465+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:53:40.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64840a11b0 msgr2=0x7f64840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:40.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64840a11b0 0x7f64840a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6488009f50 tx=0x7f6488005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64981063c0 msgr2=0x7f649819bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:40.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64981063c0 0x7f649819bb80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6490007f10 tx=0x7f649000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 shutdown_connections 2024-05-08T21:53:40.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64840a11b0 0x7f64840a3670 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64981063c0 0x7f649819bb80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64980ffba0 0x7f6498197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 --2- 172.21.15.64:0/2993620338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6498100550 0x7f64981977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 >> 172.21.15.64:0/2993620338 conn(0x7f6498075260 msgr2=0x7f64980fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 shutdown_connections 2024-05-08T21:53:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:40.387+0000 7f649e8e6700 1 -- 172.21.15.64:0/2993620338 wait complete. 2024-05-08T21:53:40.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:40 smithi186 ceph-mon[46619]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.9 KiB/s wr, 26 op/s 2024-05-08T21:53:40.411 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:40 smithi064 ceph-mon[42500]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.9 KiB/s wr, 26 op/s 2024-05-08T21:53:40.773 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T21:53:40.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T21:53:40.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T21:53:40.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T21:53:40.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T21:53:40.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T21:53:40.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T21:53:40.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T21:53:40.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T21:53:40.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T21:53:40.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T21:53:40.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T21:53:41.357 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:41.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:41 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2993620338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:41.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:41 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2993620338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:41.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:41 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2993620338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:53:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:42.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:42 smithi064 ceph-mon[42500]: pgmap v229: 97 pgs: 97 active+clean; 626 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-05-08T21:53:42.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:42 smithi186 ceph-mon[46619]: pgmap v229: 97 pgs: 97 active+clean; 626 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-05-08T21:53:42.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:42 smithi160 ceph-mon[46962]: pgmap v229: 97 pgs: 97 active+clean; 626 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 -- 172.21.15.64:0/629212771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 msgr2=0x7f92f8100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/629212771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f8100350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009990 tx=0x7f92e8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 -- 172.21.15.64:0/629212771 shutdown_connections 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/629212771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92f8100890 0x7f92f810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/629212771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f8105f00 0x7f92f81062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/629212771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f8100350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 -- 172.21.15.64:0/629212771 >> 172.21.15.64:0/629212771 conn(0x7f92f8075f80 msgr2=0x7f92f8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 -- 172.21.15.64:0/629212771 shutdown_connections 2024-05-08T21:53:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.555+0000 7f92ff8f3700 1 -- 172.21.15.64:0/629212771 wait complete. 2024-05-08T21:53:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 Processor -- start 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 -- start start 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f810d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92f8100890 0x7f92f810dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f8105f00 0x7f92f810e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81a5730 con 0x7f92f80ffed0 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81a58a0 con 0x7f92f8100890 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92ff8f3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81a5a60 con 0x7f92f8105f00 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92fd68f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f810d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92fd68f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f810d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44974/0 (socket says 172.21.15.64:44974) 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.556+0000 7f92fd68f700 1 -- 172.21.15.64:0/1680745366 learned_addr learned my addr 172.21.15.64:0/1680745366 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fce8e700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92f8100890 0x7f92f810dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fde90700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f8105f00 0x7f92f810e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fd68f700 1 -- 172.21.15.64:0/1680745366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f8105f00 msgr2=0x7f92f810e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fd68f700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f8105f00 0x7f92f810e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fd68f700 1 -- 172.21.15.64:0/1680745366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92f8100890 msgr2=0x7f92f810dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fd68f700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92f8100890 0x7f92f810dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fd68f700 1 -- 172.21.15.64:0/1680745366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80095f0 con 0x7f92f80ffed0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fde90700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f8105f00 0x7f92f810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fce8e700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92f8100890 0x7f92f810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92fd68f700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f810d710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c920 tx=0x7f92f400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4007b00 con 0x7f92f80ffed0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f4004cf0 con 0x7f92f80ffed0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4005610 con 0x7f92f80ffed0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f81a5d40 con 0x7f92f80ffed0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.557+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81a6190 con 0x7f92f80ffed0 2024-05-08T21:53:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.558+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92f81015b0 con 0x7f92f80ffed0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.559+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f92f4004750 con 0x7f92f80ffed0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92ee7fc700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f92e40a11b0 0x7f92e40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92fce8e700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f92e40a11b0 0x7f92e40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f92f40cc990 con 0x7f92f80ffed0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92ee7fc700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] conn(0x7f92e40aa990 0x7f92e40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 --> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92e40ad460 con 0x7f92e40aa990 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f40cce30 con 0x7f92f80ffed0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92fde90700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] conn(0x7f92e40aa990 0x7f92e40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.560+0000 7f92fce8e700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f92e40a11b0 0x7f92e40a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f92e8006010 tx=0x7f92e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.561+0000 7f92fde90700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] conn(0x7f92e40aa990 0x7f92e40acdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.561+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== osd.3 v2:172.21.15.64:6810/3390967060 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f92e40ad460 con 0x7f92e40aa990 2024-05-08T21:53:42.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.577+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 --> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f92f81111a0 con 0x7f92e40aa990 2024-05-08T21:53:42.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.577+0000 7f92ee7fc700 1 -- 172.21.15.64:0/1680745366 <== osd.3 v2:172.21.15.64:6810/3390967060 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f92f81111a0 con 0x7f92e40aa990 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] conn(0x7f92e40aa990 msgr2=0x7f92e40acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] conn(0x7f92e40aa990 0x7f92e40acdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f92e40a11b0 msgr2=0x7f92e40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f92e40a11b0 0x7f92e40a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f92e8006010 tx=0x7f92e801a040 comp rx=0 tx=0).stop 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 msgr2=0x7f92f810d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 --2- 172.21.15.64:0/1680745366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ffed0 0x7f92f810d710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c920 tx=0x7f92f400cce0 comp rx=0 tx=0).stop 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92fde90700 1 -- 172.21.15.64:0/1680745366 reap_dead start 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 shutdown_connections 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 >> 172.21.15.64:0/1680745366 conn(0x7f92f8075f80 msgr2=0x7f92f80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 shutdown_connections 2024-05-08T21:53:42.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:42.578+0000 7f92ff8f3700 1 -- 172.21.15.64:0/1680745366 wait complete. 2024-05-08T21:53:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:43.074 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:43.074 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:43.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:43 smithi064 ceph-mon[42500]: pgmap v230: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-05-08T21:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:43 smithi186 ceph-mon[46619]: pgmap v230: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-05-08T21:53:43.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:43 smithi160 ceph-mon[46962]: pgmap v230: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 -- 172.21.15.64:0/1669759475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c100890 msgr2=0x7fb78c10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 --2- 172.21.15.64:0/1669759475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c100890 0x7fb78c10cd40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb78400b0a0 tx=0x7fb78400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 -- 172.21.15.64:0/1669759475 shutdown_connections 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 --2- 172.21.15.64:0/1669759475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c100890 0x7fb78c10cd40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 --2- 172.21.15.64:0/1669759475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c105f00 0x7fb78c1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 --2- 172.21.15.64:0/1669759475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb78c0ffed0 0x7fb78c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 -- 172.21.15.64:0/1669759475 >> 172.21.15.64:0/1669759475 conn(0x7fb78c075f80 msgr2=0x7fb78c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 -- 172.21.15.64:0/1669759475 shutdown_connections 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.655+0000 7fb79086b700 1 -- 172.21.15.64:0/1669759475 wait complete. 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 Processor -- start 2024-05-08T21:53:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 -- start start 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c0ffed0 0x7fb78c10d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb78c100890 0x7fb78c10dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c105f00 0x7fb78c10e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a57d0 con 0x7fb78c0ffed0 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a5940 con 0x7fb78c100890 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb79086b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a5b00 con 0x7fb78c105f00 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c0ffed0 0x7fb78c10d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c0ffed0 0x7fb78c10d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44990/0 (socket says 172.21.15.64:44990) 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 -- 172.21.15.64:0/1683468248 learned_addr learned my addr 172.21.15.64:0/1683468248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb789d9b700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb78c100890 0x7fb78c10dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78ad9d700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c105f00 0x7fb78c10e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 -- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c105f00 msgr2=0x7fb78c10e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c105f00 0x7fb78c10e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 -- 172.21.15.64:0/1683468248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb78c100890 msgr2=0x7fb78c10dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.656+0000 7fb78a59c700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb78c100890 0x7fb78c10dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb78a59c700 1 -- 172.21.15.64:0/1683468248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784009d00 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb78a59c700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c0ffed0 0x7fb78c10d710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c9b0 tx=0x7fb77c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007b00 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb77c007c60 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c1a5de0 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c018660 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.657+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c1a6260 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.658+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb78c1111c0 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.659+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb77c00f450 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.659+0000 7fb77b7fe700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7740a1230 0x7fb7740a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb789d9b700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7740a1230 0x7fb7740a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb77c0ccc80 con 0x7fb78c0ffed0 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb77b7fe700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] conn(0x7fb7740aaa10 0x7fb7740ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 --> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7740ad4e0 con 0x7fb7740aaa10 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb77c0ccff0 con 0x7fb78c0ffed0 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb789d9b700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7740a1230 0x7fb7740a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb78c10ee40 tx=0x7fb780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb78ad9d700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] conn(0x7fb7740aaa10 0x7fb7740ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.660+0000 7fb78ad9d700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] conn(0x7fb7740aaa10 0x7fb7740ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.661+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== osd.8 v2:172.21.15.186:6816/4072780638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb7740ad4e0 con 0x7fb7740aaa10 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.677+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 --> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb78c1015b0 con 0x7fb7740aaa10 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb77b7fe700 1 -- 172.21.15.64:0/1683468248 <== osd.8 v2:172.21.15.186:6816/4072780638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb78c1015b0 con 0x7fb7740aaa10 2024-05-08T21:53:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] conn(0x7fb7740aaa10 msgr2=0x7fb7740ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] conn(0x7fb7740aaa10 0x7fb7740ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7740a1230 msgr2=0x7fb7740a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7740a1230 0x7fb7740a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb78c10ee40 tx=0x7fb780009260 comp rx=0 tx=0).stop 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c0ffed0 msgr2=0x7fb78c10d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 --2- 172.21.15.64:0/1683468248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb78c0ffed0 0x7fb78c10d710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c9b0 tx=0x7fb77c00ccc0 comp rx=0 tx=0).stop 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb78ad9d700 1 -- 172.21.15.64:0/1683468248 reap_dead start 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 shutdown_connections 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 >> 172.21.15.64:0/1683468248 conn(0x7fb78c075f80 msgr2=0x7fb78c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 shutdown_connections 2024-05-08T21:53:44.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:43.678+0000 7fb79086b700 1 -- 172.21.15.64:0/1683468248 wait complete. 2024-05-08T21:53:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:103079215147 2024-05-08T21:53:44.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T21:53:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:44.559 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:44.998 INFO:teuthology.orchestra.run.smithi064.stdout:236223201305 2024-05-08T21:53:44.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T21:53:45.458 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:45 smithi064 ceph-mon[42500]: pgmap v231: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 16 op/s 2024-05-08T21:53:45.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:45 smithi160 ceph-mon[46962]: pgmap v231: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 16 op/s 2024-05-08T21:53:45.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:45 smithi186 ceph-mon[46619]: pgmap v231: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 16 op/s 2024-05-08T21:53:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 -- 172.21.15.64:0/1044150486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898105f00 msgr2=0x7fc8981062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 --2- 172.21.15.64:0/1044150486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898105f00 0x7fc8981062e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc888009990 tx=0x7fc888009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 -- 172.21.15.64:0/1044150486 shutdown_connections 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 --2- 172.21.15.64:0/1044150486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc898100890 0x7fc89810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 --2- 172.21.15.64:0/1044150486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898105f00 0x7fc8981062e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 --2- 172.21.15.64:0/1044150486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8980ffed0 0x7fc898100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 -- 172.21.15.64:0/1044150486 >> 172.21.15.64:0/1044150486 conn(0x7fc898075f80 msgr2=0x7fc898076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 -- 172.21.15.64:0/1044150486 shutdown_connections 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.836+0000 7fc89e56b700 1 -- 172.21.15.64:0/1044150486 wait complete. 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 Processor -- start 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 -- start start 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8980ffed0 0x7fc89819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc898100890 0x7fc89819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898105f00 0x7fc898195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc898195900 con 0x7fc8980ffed0 2024-05-08T21:53:46.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc898195a70 con 0x7fc898105f00 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.837+0000 7fc89e56b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc898195c30 con 0x7fc898100890 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898105f00 0x7fc898195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898105f00 0x7fc898195130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:46220/0 (socket says 172.21.15.64:46220) 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 -- 172.21.15.64:0/3947240971 learned_addr learned my addr 172.21.15.64:0/3947240971 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc8977fe700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc898100890 0x7fc89819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc897fff700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8980ffed0 0x7fc89819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 -- 172.21.15.64:0/3947240971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc898100890 msgr2=0x7fc89819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc898100890 0x7fc89819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 -- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8980ffed0 msgr2=0x7fc89819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8980ffed0 0x7fc89819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 -- 172.21.15.64:0/3947240971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880095f0 con 0x7fc898105f00 2024-05-08T21:53:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.838+0000 7fc89cb08700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898105f00 0x7fc898195130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc890007ee0 tx=0x7fc89000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.839+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890017070 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.839+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89000f040 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.839+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890005580 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.839+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc898199b20 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.839+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc898199fd0 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.839+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc89810f000 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.841+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc89000db20 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.842+0000 7fc8957fa700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc8800a12e0 0x7fc8800a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.842+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8900cc680 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.842+0000 7fc897fff700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc8800a12e0 0x7fc8800a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.842+0000 7fc8957fa700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] conn(0x7fc8800aaac0 0x7fc8800acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.842+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 --> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8800ad590 con 0x7fc8800aaac0 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.842+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8900ccaa0 con 0x7fc898105f00 2024-05-08T21:53:46.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.843+0000 7fc897fff700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc8800a12e0 0x7fc8800a37a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc888003760 tx=0x7fc88800b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.843+0000 7fc8977fe700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] conn(0x7fc8800aaac0 0x7fc8800acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.843+0000 7fc8977fe700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] conn(0x7fc8800aaac0 0x7fc8800acee0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.844+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== osd.4 v2:172.21.15.160:6808/1821714303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc8800ad590 con 0x7fc8800aaac0 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.860+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 --> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc8981015b0 con 0x7fc8800aaac0 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc8957fa700 1 -- 172.21.15.64:0/3947240971 <== osd.4 v2:172.21.15.160:6808/1821714303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc8981015b0 con 0x7fc8800aaac0 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] conn(0x7fc8800aaac0 msgr2=0x7fc8800acee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] conn(0x7fc8800aaac0 0x7fc8800acee0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc8800a12e0 msgr2=0x7fc8800a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc8800a12e0 0x7fc8800a37a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc888003760 tx=0x7fc88800b580 comp rx=0 tx=0).stop 2024-05-08T21:53:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898105f00 msgr2=0x7fc898195130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:46.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 --2- 172.21.15.64:0/3947240971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898105f00 0x7fc898195130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc890007ee0 tx=0x7fc89000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:46.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89cb08700 1 -- 172.21.15.64:0/3947240971 reap_dead start 2024-05-08T21:53:46.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 shutdown_connections 2024-05-08T21:53:46.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 >> 172.21.15.64:0/3947240971 conn(0x7fc898075f80 msgr2=0x7fc8980fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:46.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 shutdown_connections 2024-05-08T21:53:46.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:45.861+0000 7fc89e56b700 1 -- 172.21.15.64:0/3947240971 wait complete. 2024-05-08T21:53:46.214 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:47.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:47 smithi064 ceph-mon[42500]: pgmap v232: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-05-08T21:53:47.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:47 smithi160 ceph-mon[46962]: pgmap v232: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 -- 172.21.15.64:0/1184275474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11700ffed0 msgr2=0x7f1170100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1184275474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11700ffed0 0x7f1170100350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1164009990 tx=0x7f1164009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 -- 172.21.15.64:0/1184275474 shutdown_connections 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1184275474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1170100890 0x7f117010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1184275474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170105f00 0x7f11701062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1184275474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11700ffed0 0x7f1170100350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 -- 172.21.15.64:0/1184275474 >> 172.21.15.64:0/1184275474 conn(0x7f1170075f80 msgr2=0x7f1170076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 -- 172.21.15.64:0/1184275474 shutdown_connections 2024-05-08T21:53:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.564+0000 7f1174d0c700 1 -- 172.21.15.64:0/1184275474 wait complete. 2024-05-08T21:53:47.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.565+0000 7f1174d0c700 1 Processor -- start 2024-05-08T21:53:47.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 -- start start 2024-05-08T21:53:47.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11700ffed0 0x7f1170101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:47.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1170100890 0x7f11701020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170105f00 0x7f11701025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11701a58f0 con 0x7f1170100890 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11701a5a60 con 0x7f11700ffed0 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f1174d0c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11701a5c20 con 0x7f1170105f00 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1170100890 0x7f11701020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1170100890 0x7f11701020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45038/0 (socket says 172.21.15.64:45038) 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 -- 172.21.15.64:0/1788058734 learned_addr learned my addr 172.21.15.64:0/1788058734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116e59c700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11700ffed0 0x7f1170101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116ed9d700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170105f00 0x7f11701025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 -- 172.21.15.64:0/1788058734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170105f00 msgr2=0x7f11701025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170105f00 0x7f11701025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 -- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11700ffed0 msgr2=0x7f1170101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11700ffed0 0x7f1170101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116dd9b700 1 -- 172.21.15.64:0/1788058734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640095f0 con 0x7f1170100890 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.566+0000 7f116ed9d700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170105f00 0x7f11701025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.567+0000 7f116dd9b700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1170100890 0x7f11701020b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11640098e0 tx=0x7f116400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.567+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116401d070 con 0x7f1170100890 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.567+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1164022470 con 0x7f1170100890 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.567+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11701a5ea0 con 0x7f1170100890 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.567+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164017650 con 0x7f1170100890 2024-05-08T21:53:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.567+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11701a62b0 con 0x7f1170100890 2024-05-08T21:53:47.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.568+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1170110070 con 0x7f1170100890 2024-05-08T21:53:47.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.569+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f11640225e0 con 0x7f1170100890 2024-05-08T21:53:47.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.570+0000 7f115f7fe700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f11580a1230 0x7f11580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:47.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.570+0000 7f116e59c700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f11580a1230 0x7f11580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.570+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f11640ce110 con 0x7f1170100890 2024-05-08T21:53:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.570+0000 7f115f7fe700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] conn(0x7f11580aaa10 0x7f11580ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.570+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 --> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11580ad4e0 con 0x7f11580aaa10 2024-05-08T21:53:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.570+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11640ce480 con 0x7f1170100890 2024-05-08T21:53:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.571+0000 7f116e59c700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f11580a1230 0x7f11580a36f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1160009a10 tx=0x7f1160009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.571+0000 7f116ed9d700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] conn(0x7f11580aaa10 0x7f11580ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.571+0000 7f116ed9d700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] conn(0x7f11580aaa10 0x7f11580ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.572+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== osd.7 v2:172.21.15.160:6816/3316502570 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f11580ad4e0 con 0x7f11580aaa10 2024-05-08T21:53:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.596+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 --> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1170104f00 con 0x7f11580aaa10 2024-05-08T21:53:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.596+0000 7f115f7fe700 1 -- 172.21.15.64:0/1788058734 <== osd.7 v2:172.21.15.160:6816/3316502570 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1170104f00 con 0x7f11580aaa10 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] conn(0x7f11580aaa10 msgr2=0x7f11580ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] conn(0x7f11580aaa10 0x7f11580ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f11580a1230 msgr2=0x7f11580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f11580a1230 0x7f11580a36f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1160009a10 tx=0x7f1160009190 comp rx=0 tx=0).stop 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1170100890 msgr2=0x7f11701020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 --2- 172.21.15.64:0/1788058734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1170100890 0x7f11701020b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11640098e0 tx=0x7f116400f670 comp rx=0 tx=0).stop 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f116ed9d700 1 -- 172.21.15.64:0/1788058734 reap_dead start 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 shutdown_connections 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 >> 172.21.15.64:0/1788058734 conn(0x7f1170075f80 msgr2=0x7f1170068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 shutdown_connections 2024-05-08T21:53:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:47.597+0000 7f1174d0c700 1 -- 172.21.15.64:0/1788058734 wait complete. 2024-05-08T21:53:47.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:47 smithi186 ceph-mon[46619]: pgmap v232: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-05-08T21:53:48.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 -- 172.21.15.64:0/3396011554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 msgr2=0x7f34c01062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 --2- 172.21.15.64:0/3396011554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01062e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009990 tx=0x7f34b0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 -- 172.21.15.64:0/3396011554 shutdown_connections 2024-05-08T21:53:48.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 --2- 172.21.15.64:0/3396011554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34c0100890 0x7f34c010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 --2- 172.21.15.64:0/3396011554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01062e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 --2- 172.21.15.64:0/3396011554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c00ffed0 0x7f34c0100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 -- 172.21.15.64:0/3396011554 >> 172.21.15.64:0/3396011554 conn(0x7f34c0075f80 msgr2=0x7f34c0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:48.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 -- 172.21.15.64:0/3396011554 shutdown_connections 2024-05-08T21:53:48.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.228+0000 7f34c522b700 1 -- 172.21.15.64:0/3396011554 wait complete. 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 Processor -- start 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 -- start start 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34c00ffed0 0x7f34c0101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0100890 0x7f34c01020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c01a57e0 con 0x7f34c0105f00 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c01a5950 con 0x7f34c00ffed0 2024-05-08T21:53:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34c522b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c01a5b10 con 0x7f34c0100890 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.229+0000 7f34bf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45060/0 (socket says 172.21.15.64:45060) 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 -- 172.21.15.64:0/368099134 learned_addr learned my addr 172.21.15.64:0/368099134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bed9d700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34c00ffed0 0x7f34c0101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34be59c700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0100890 0x7f34c01020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 -- 172.21.15.64:0/368099134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0100890 msgr2=0x7f34c01020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0100890 0x7f34c01020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 -- 172.21.15.64:0/368099134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34c00ffed0 msgr2=0x7f34c0101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34c00ffed0 0x7f34c0101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 -- 172.21.15.64:0/368099134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00095f0 con 0x7f34c0105f00 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34bf59e700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01025f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34b800ada0 tx=0x7f34b800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b800f070 con 0x7f34c0105f00 2024-05-08T21:53:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c01a5df0 con 0x7f34c0105f00 2024-05-08T21:53:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b8007b30 con 0x7f34c0105f00 2024-05-08T21:53:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80053e0 con 0x7f34c0105f00 2024-05-08T21:53:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.230+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c01a6270 con 0x7f34c0105f00 2024-05-08T21:53:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.231+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34c0110080 con 0x7f34c0105f00 2024-05-08T21:53:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f34b800c980 con 0x7f34c0105f00 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34a7fff700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34a80a11b0 0x7f34a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f34b80cc800 con 0x7f34c0105f00 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34bed9d700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34a80a11b0 0x7f34a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34a7fff700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] conn(0x7f34a80aa990 0x7f34a80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 --> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34a80ad460 con 0x7f34a80aa990 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.233+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b80ccca0 con 0x7f34c0105f00 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.234+0000 7f34be59c700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] conn(0x7f34a80aa990 0x7f34a80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.234+0000 7f34bed9d700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34a80a11b0 0x7f34a80a3670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b5c0 tx=0x7f34b0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.234+0000 7f34be59c700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] conn(0x7f34a80aa990 0x7f34a80acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:48.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.235+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== osd.9 v2:172.21.15.64:6826/200787259 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f34a80ad460 con 0x7f34a80aa990 2024-05-08T21:53:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.251+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 --> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34c01015b0 con 0x7f34a80aa990 2024-05-08T21:53:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.251+0000 7f34a7fff700 1 -- 172.21.15.64:0/368099134 <== osd.9 v2:172.21.15.64:6826/200787259 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f34c01015b0 con 0x7f34a80aa990 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] conn(0x7f34a80aa990 msgr2=0x7f34a80acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] conn(0x7f34a80aa990 0x7f34a80acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34a80a11b0 msgr2=0x7f34a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34a80a11b0 0x7f34a80a3670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b5c0 tx=0x7f34b0005960 comp rx=0 tx=0).stop 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 msgr2=0x7f34c01025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 --2- 172.21.15.64:0/368099134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0105f00 0x7f34c01025f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34b800ada0 tx=0x7f34b800c330 comp rx=0 tx=0).stop 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34bf59e700 1 -- 172.21.15.64:0/368099134 reap_dead start 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 shutdown_connections 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 >> 172.21.15.64:0/368099134 conn(0x7f34c0075f80 msgr2=0x7f34c0068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 shutdown_connections 2024-05-08T21:53:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.252+0000 7f34c522b700 1 -- 172.21.15.64:0/368099134 wait complete. 2024-05-08T21:53:48.663 INFO:teuthology.orchestra.run.smithi064.stdout:133143986216 2024-05-08T21:53:48.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T21:53:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 -- 172.21.15.64:0/4231163726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b140692f0 msgr2=0x7f1b1410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 --2- 172.21.15.64:0/4231163726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b140692f0 0x7f1b1410de60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b0a0 tx=0x7f1b0c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 -- 172.21.15.64:0/4231163726 shutdown_connections 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 --2- 172.21.15.64:0/4231163726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14107020 0x7f1b14107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 --2- 172.21.15.64:0/4231163726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b140692f0 0x7f1b1410de60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 --2- 172.21.15.64:0/4231163726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b14068950 0x7f1b14068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 -- 172.21.15.64:0/4231163726 >> 172.21.15.64:0/4231163726 conn(0x7f1b14075f80 msgr2=0x7f1b14076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 -- 172.21.15.64:0/4231163726 shutdown_connections 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.881+0000 7f1b1936f700 1 -- 172.21.15.64:0/4231163726 wait complete. 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 Processor -- start 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 -- start start 2024-05-08T21:53:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b14068950 0x7f1b14197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b140692f0 0x7f1b14197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14107020 0x7f1b1419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1419c3b0 con 0x7f1b14068950 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1419c520 con 0x7f1b14107020 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.882+0000 7f1b1936f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1419c6e0 con 0x7f1b140692f0 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b14068950 0x7f1b14197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b14068950 0x7f1b14197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45086/0 (socket says 172.21.15.64:45086) 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 -- 172.21.15.64:0/332709718 learned_addr learned my addr 172.21.15.64:0/332709718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b127fc700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b140692f0 0x7f1b14197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b137fe700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14107020 0x7f1b1419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 -- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b140692f0 msgr2=0x7f1b14197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b140692f0 0x7f1b14197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 -- 172.21.15.64:0/332709718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14107020 msgr2=0x7f1b1419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14107020 0x7f1b1419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 -- 172.21.15.64:0/332709718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c009d00 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b12ffd700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b14068950 0x7f1b14197500 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400d980 tx=0x7f1b0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004220 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b04007640 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04005690 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1419c9c0 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.883+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b141a4220 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.884+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b14111150 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.885+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1b04004750 con 0x7f1b14068950 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1afbfff700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1afc0a11b0 0x7f1afc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1b127fc700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1afc0a11b0 0x7f1afc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b040ccc80 con 0x7f1b14068950 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1afbfff700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] conn(0x7f1afc0aa990 0x7f1afc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 --> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1afc0ad460 con 0x7f1afc0aa990 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b040cd120 con 0x7f1b14068950 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1b127fc700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1afc0a11b0 0x7f1afc0a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1b14198a10 tx=0x7f1b0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.886+0000 7f1b137fe700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] conn(0x7f1afc0aa990 0x7f1afc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.887+0000 7f1b137fe700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] conn(0x7f1afc0aa990 0x7f1afc0acdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.888+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== osd.5 v2:172.21.15.186:6808/1348911288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1afc0ad460 con 0x7f1afc0aa990 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.904+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 --> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b141026d0 con 0x7f1afc0aa990 2024-05-08T21:53:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1afbfff700 1 -- 172.21.15.64:0/332709718 <== osd.5 v2:172.21.15.186:6808/1348911288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1b141026d0 con 0x7f1afc0aa990 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] conn(0x7f1afc0aa990 msgr2=0x7f1afc0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] conn(0x7f1afc0aa990 0x7f1afc0acdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1afc0a11b0 msgr2=0x7f1afc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1afc0a11b0 0x7f1afc0a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1b14198a10 tx=0x7f1b0800b410 comp rx=0 tx=0).stop 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b14068950 msgr2=0x7f1b14197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 --2- 172.21.15.64:0/332709718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b14068950 0x7f1b14197500 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400d980 tx=0x7f1b0400dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b137fe700 1 -- 172.21.15.64:0/332709718 reap_dead start 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 shutdown_connections 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.905+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 >> 172.21.15.64:0/332709718 conn(0x7f1b14075f80 msgr2=0x7f1b140fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.906+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 shutdown_connections 2024-05-08T21:53:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:48.906+0000 7f1b1936f700 1 -- 172.21.15.64:0/332709718 wait complete. 2024-05-08T21:53:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:49 smithi064 ceph-mon[42500]: pgmap v233: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-08T21:53:49.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:49 smithi160 ceph-mon[46962]: pgmap v233: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-08T21:53:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 -- 172.21.15.64:0/1264831970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde68104650 msgr2=0x7fde68104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 --2- 172.21.15.64:0/1264831970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde68104650 0x7fde68104a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fde58009990 tx=0x7fde58009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 -- 172.21.15.64:0/1264831970 shutdown_connections 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 --2- 172.21.15.64:0/1264831970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde68100300 0x7fde6810c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 --2- 172.21.15.64:0/1264831970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde68104650 0x7fde68104a30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 --2- 172.21.15.64:0/1264831970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde680ff940 0x7fde680ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 -- 172.21.15.64:0/1264831970 >> 172.21.15.64:0/1264831970 conn(0x7fde68075260 msgr2=0x7fde68075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 -- 172.21.15.64:0/1264831970 shutdown_connections 2024-05-08T21:53:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.547+0000 7fde6d246700 1 -- 172.21.15.64:0/1264831970 wait complete. 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 Processor -- start 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 -- start start 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680ff940 0x7fde68197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde68100300 0x7fde68197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde68104650 0x7fde6819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6819c1f0 con 0x7fde680ff940 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6819c360 con 0x7fde68100300 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde6d246700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6819c520 con 0x7fde68104650 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde66d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680ff940 0x7fde68197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde66d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680ff940 0x7fde68197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45092/0 (socket says 172.21.15.64:45092) 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.548+0000 7fde66d9d700 1 -- 172.21.15.64:0/1510149072 learned_addr learned my addr 172.21.15.64:0/1510149072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde6659c700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde68100300 0x7fde68197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde6759e700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde68104650 0x7fde6819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde66d9d700 1 -- 172.21.15.64:0/1510149072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde68104650 msgr2=0x7fde6819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde66d9d700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde68104650 0x7fde6819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde66d9d700 1 -- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde68100300 msgr2=0x7fde68197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde66d9d700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde68100300 0x7fde68197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde66d9d700 1 -- 172.21.15.64:0/1510149072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580095f0 con 0x7fde680ff940 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde6659c700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde68100300 0x7fde68197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde66d9d700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680ff940 0x7fde68197340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde580098e0 tx=0x7fde5800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5801d070 con 0x7fde680ff940 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6819c7a0 con 0x7fde680ff940 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde58005590 con 0x7fde680ff940 2024-05-08T21:53:49.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58003de0 con 0x7fde680ff940 2024-05-08T21:53:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.549+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde681a4060 con 0x7fde680ff940 2024-05-08T21:53:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.550+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde68100d60 con 0x7fde680ff940 2024-05-08T21:53:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.552+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fde5800fc40 con 0x7fde680ff940 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.552+0000 7fde4ffff700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fde500a11b0 0x7fde500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.552+0000 7fde6659c700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fde500a11b0 0x7fde500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.552+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fde580cda60 con 0x7fde680ff940 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.553+0000 7fde4ffff700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] conn(0x7fde500aa860 0x7fde500acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.553+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 --> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde500ad330 con 0x7fde500aa860 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.553+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde580cdf00 con 0x7fde680ff940 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.553+0000 7fde6659c700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fde500a11b0 0x7fde500a3670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005fd0 tx=0x7fde5c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.553+0000 7fde6759e700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] conn(0x7fde500aa860 0x7fde500acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.553+0000 7fde6759e700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] conn(0x7fde500aa860 0x7fde500acc80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.554+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== osd.10 v2:172.21.15.160:6824/1117750760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fde500ad330 con 0x7fde500aa860 2024-05-08T21:53:49.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.570+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 --> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde6810ef90 con 0x7fde500aa860 2024-05-08T21:53:49.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde4ffff700 1 -- 172.21.15.64:0/1510149072 <== osd.10 v2:172.21.15.160:6824/1117750760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fde6810ef90 con 0x7fde500aa860 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] conn(0x7fde500aa860 msgr2=0x7fde500acc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] conn(0x7fde500aa860 0x7fde500acc80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fde500a11b0 msgr2=0x7fde500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fde500a11b0 0x7fde500a3670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005fd0 tx=0x7fde5c005e20 comp rx=0 tx=0).stop 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680ff940 msgr2=0x7fde68197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 --2- 172.21.15.64:0/1510149072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680ff940 0x7fde68197340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde580098e0 tx=0x7fde5800f740 comp rx=0 tx=0).stop 2024-05-08T21:53:49.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6759e700 1 -- 172.21.15.64:0/1510149072 reap_dead start 2024-05-08T21:53:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 shutdown_connections 2024-05-08T21:53:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 >> 172.21.15.64:0/1510149072 conn(0x7fde68075260 msgr2=0x7fde680fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 shutdown_connections 2024-05-08T21:53:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.571+0000 7fde6d246700 1 -- 172.21.15.64:0/1510149072 wait complete. 2024-05-08T21:53:49.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:49 smithi186 ceph-mon[46619]: pgmap v233: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-08T21:53:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 -- 172.21.15.64:0/3769235872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c100a50 msgr2=0x7f971c104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 --2- 172.21.15.64:0/3769235872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c100a50 0x7f971c104aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f970c009990 tx=0x7f970c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 -- 172.21.15.64:0/3769235872 shutdown_connections 2024-05-08T21:53:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 --2- 172.21.15.64:0/3769235872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f971c1000a0 0x7f971c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 --2- 172.21.15.64:0/3769235872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f971c105180 0x7f971c10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 --2- 172.21.15.64:0/3769235872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c100a50 0x7f971c104aa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 -- 172.21.15.64:0/3769235872 >> 172.21.15.64:0/3769235872 conn(0x7f971c076ea0 msgr2=0x7f971c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:49.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 -- 172.21.15.64:0/3769235872 shutdown_connections 2024-05-08T21:53:49.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 -- 172.21.15.64:0/3769235872 wait complete. 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.978+0000 7f97216f8700 1 Processor -- start 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 -- start start 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c1000a0 0x7f971c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f971c100a50 0x7f971c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f971c105180 0x7f971c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19c2d0 con 0x7f971c1000a0 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19c440 con 0x7f971c100a50 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f97216f8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19c600 con 0x7f971c105180 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c1000a0 0x7f971c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c1000a0 0x7f971c1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45114/0 (socket says 172.21.15.64:45114) 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 -- 172.21.15.64:0/2582133835 learned_addr learned my addr 172.21.15.64:0/2582133835 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971a7fc700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f971c100a50 0x7f971c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971b7fe700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f971c105180 0x7f971c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 -- 172.21.15.64:0/2582133835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f971c105180 msgr2=0x7f971c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f971c105180 0x7f971c19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 -- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f971c100a50 msgr2=0x7f971c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f971c100a50 0x7f971c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 -- 172.21.15.64:0/2582133835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c0095f0 con 0x7f971c1000a0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.979+0000 7f971affd700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c1000a0 0x7f971c1973d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f970400e9a0 tx=0x7f970400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.980+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704009790 con 0x7f971c1000a0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.980+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c19c8e0 con 0x7f971c1000a0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.980+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970400cbd0 con 0x7f971c1000a0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.980+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704018680 con 0x7f971c1000a0 2024-05-08T21:53:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.980+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c101a70 con 0x7f971c1000a0 2024-05-08T21:53:49.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.980+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f971c101580 con 0x7f971c1000a0 2024-05-08T21:53:49.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.982+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f970400cd40 con 0x7f971c1000a0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.982+0000 7f9713fff700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97080a55c0 0x7f97080a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.982+0000 7f971a7fc700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97080a55c0 0x7f97080a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.982+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9704014070 con 0x7f971c1000a0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.983+0000 7f9713fff700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] conn(0x7f97080aeda0 0x7f97080b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.983+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 --> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97080b1870 con 0x7f97080aeda0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.983+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97040cd3b0 con 0x7f971c1000a0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.983+0000 7f971b7fe700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] conn(0x7f97080aeda0 0x7f97080b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.983+0000 7f971a7fc700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97080a55c0 0x7f97080a7a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b2e0 tx=0x7f970c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.983+0000 7f971b7fe700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] conn(0x7f97080aeda0 0x7f97080b11c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:49.984+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== osd.1 v2:172.21.15.160:6800/255573716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f97080b1870 con 0x7f97080aeda0 2024-05-08T21:53:50.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.000+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 --> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f971c10fff0 con 0x7f97080aeda0 2024-05-08T21:53:50.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f9713fff700 1 -- 172.21.15.64:0/2582133835 <== osd.1 v2:172.21.15.160:6800/255573716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f971c10fff0 con 0x7f97080aeda0 2024-05-08T21:53:50.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] conn(0x7f97080aeda0 msgr2=0x7f97080b11c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] conn(0x7f97080aeda0 0x7f97080b11c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97080a55c0 msgr2=0x7f97080a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97080a55c0 0x7f97080a7a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b2e0 tx=0x7f970c005960 comp rx=0 tx=0).stop 2024-05-08T21:53:50.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c1000a0 msgr2=0x7f971c1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 --2- 172.21.15.64:0/2582133835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c1000a0 0x7f971c1973d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f970400e9a0 tx=0x7f970400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f971b7fe700 1 -- 172.21.15.64:0/2582133835 reap_dead start 2024-05-08T21:53:50.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 shutdown_connections 2024-05-08T21:53:50.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.001+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 >> 172.21.15.64:0/2582133835 conn(0x7f971c076ea0 msgr2=0x7f971c069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:50.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.002+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 shutdown_connections 2024-05-08T21:53:50.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.002+0000 7f97216f8700 1 -- 172.21.15.64:0/2582133835 wait complete. 2024-05-08T21:53:50.213 INFO:teuthology.orchestra.run.smithi064.stdout:266287972374 2024-05-08T21:53:50.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T21:53:50.281 INFO:teuthology.orchestra.run.smithi064.stdout:206158430236 2024-05-08T21:53:50.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.570+0000 7f3472fb8700 1 -- 172.21.15.64:0/1920265314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c069bd0 msgr2=0x7f346c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.570+0000 7f3472fb8700 1 --2- 172.21.15.64:0/1920265314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c069bd0 0x7f346c10de00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f346400b0d0 tx=0x7f346400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 -- 172.21.15.64:0/1920265314 shutdown_connections 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 --2- 172.21.15.64:0/1920265314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f346c106fe0 0x7f346c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 --2- 172.21.15.64:0/1920265314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c069bd0 0x7f346c10de00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 --2- 172.21.15.64:0/1920265314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f346c069210 0x7f346c069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 -- 172.21.15.64:0/1920265314 >> 172.21.15.64:0/1920265314 conn(0x7f346c076e70 msgr2=0x7f346c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 -- 172.21.15.64:0/1920265314 shutdown_connections 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 -- 172.21.15.64:0/1920265314 wait complete. 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 Processor -- start 2024-05-08T21:53:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.571+0000 7f3472fb8700 1 -- start start 2024-05-08T21:53:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f346c069210 0x7f346c1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f346c069bd0 0x7f346c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c106fe0 0x7f346c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19c2f0 con 0x7f346c106fe0 2024-05-08T21:53:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19c460 con 0x7f346c069210 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19c620 con 0x7f346c069bd0 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c106fe0 0x7f346c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c106fe0 0x7f346c19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45132/0 (socket says 172.21.15.64:45132) 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 -- 172.21.15.64:0/38870922 learned_addr learned my addr 172.21.15.64:0/38870922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3470d54700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f346c069210 0x7f346c1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f346bfff700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f346c069bd0 0x7f346c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 -- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f346c069bd0 msgr2=0x7f346c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f346c069bd0 0x7f346c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 -- 172.21.15.64:0/38870922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f346c069210 msgr2=0x7f346c1973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f346c069210 0x7f346c1973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 -- 172.21.15.64:0/38870922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464009d00 con 0x7f346c106fe0 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3471555700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c106fe0 0x7f346c19bce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34640094c0 tx=0x7f34640094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400e040 con 0x7f346c106fe0 2024-05-08T21:53:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c19c8a0 con 0x7f346c106fe0 2024-05-08T21:53:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f346401d940 con 0x7f346c106fe0 2024-05-08T21:53:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.572+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346401b9e0 con 0x7f346c106fe0 2024-05-08T21:53:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.573+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c1a4160 con 0x7f346c106fe0 2024-05-08T21:53:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.574+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f346c102690 con 0x7f346c106fe0 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.575+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f346401d480 con 0x7f346c106fe0 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.575+0000 7f3469ffb700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f345c0a11b0 0x7f345c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.575+0000 7f3470d54700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f345c0a11b0 0x7f345c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.575+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f34640cea70 con 0x7f346c106fe0 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.576+0000 7f3469ffb700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] conn(0x7f345c0aa990 0x7f345c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.576+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 --> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f345c0ad460 con 0x7f345c0aa990 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.576+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34640cef10 con 0x7f346c106fe0 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.576+0000 7f3470d54700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f345c0a11b0 0x7f345c0a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3460005950 tx=0x7f346000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.576+0000 7f346bfff700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] conn(0x7f345c0aa990 0x7f345c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:50.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.576+0000 7f346bfff700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] conn(0x7f345c0aa990 0x7f345c0acdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:50.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.577+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== osd.11 v2:172.21.15.186:6824/2726552436 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f345c0ad460 con 0x7f345c0aa990 2024-05-08T21:53:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.594+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 --> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f346c111160 con 0x7f345c0aa990 2024-05-08T21:53:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.594+0000 7f3469ffb700 1 -- 172.21.15.64:0/38870922 <== osd.11 v2:172.21.15.186:6824/2726552436 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f346c111160 con 0x7f345c0aa990 2024-05-08T21:53:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.595+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] conn(0x7f345c0aa990 msgr2=0x7f345c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.595+0000 7f3472fb8700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] conn(0x7f345c0aa990 0x7f345c0acdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f345c0a11b0 msgr2=0x7f345c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f345c0a11b0 0x7f345c0a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3460005950 tx=0x7f346000b410 comp rx=0 tx=0).stop 2024-05-08T21:53:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c106fe0 msgr2=0x7f346c19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 --2- 172.21.15.64:0/38870922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f346c106fe0 0x7f346c19bce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34640094c0 tx=0x7f34640094f0 comp rx=0 tx=0).stop 2024-05-08T21:53:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3471555700 1 -- 172.21.15.64:0/38870922 reap_dead start 2024-05-08T21:53:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 shutdown_connections 2024-05-08T21:53:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 >> 172.21.15.64:0/38870922 conn(0x7f346c076e70 msgr2=0x7f346c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 shutdown_connections 2024-05-08T21:53:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:50.596+0000 7f3472fb8700 1 -- 172.21.15.64:0/38870922 wait complete. 2024-05-08T21:53:51.296 INFO:teuthology.orchestra.run.smithi064.stdout:55834574897 2024-05-08T21:53:51.296 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T21:53:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/1223027690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48100550 msgr2=0x7f6f48105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/1223027690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48100550 0x7f6f48105c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f380099c0 tx=0x7f6f38009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/1223027690 shutdown_connections 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/1223027690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f480ffba0 0x7f6f480fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/1223027690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f481063c0 0x7f6f4810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/1223027690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48100550 0x7f6f48105c90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/1223027690 >> 172.21.15.64:0/1223027690 conn(0x7f6f48075260 msgr2=0x7f6f48075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/1223027690 shutdown_connections 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.177+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/1223027690 wait complete. 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 Processor -- start 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 -- start start 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f480ffba0 0x7f6f48197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f48100550 0x7f6f48197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f481063c0 0x7f6f4819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4819c200 con 0x7f6f480ffba0 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4819c370 con 0x7f6f48100550 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4f8d3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4819c530 con 0x7f6f481063c0 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4d66f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f480ffba0 0x7f6f48197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4d66f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f480ffba0 0x7f6f48197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45174/0 (socket says 172.21.15.64:45174) 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4d66f700 1 -- 172.21.15.64:0/175202862 learned_addr learned my addr 172.21.15.64:0/175202862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4de70700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f481063c0 0x7f6f4819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.178+0000 7f6f4ce6e700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f48100550 0x7f6f48197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4d66f700 1 -- 172.21.15.64:0/175202862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f481063c0 msgr2=0x7f6f4819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4d66f700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f481063c0 0x7f6f4819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4d66f700 1 -- 172.21.15.64:0/175202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f48100550 msgr2=0x7f6f48197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4d66f700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f48100550 0x7f6f48197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4d66f700 1 -- 172.21.15.64:0/175202862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f380095f0 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4d66f700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f480ffba0 0x7f6f48197300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400d950 tx=0x7f6f4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400f890 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4819c810 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f44010460 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400e610 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.179+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f481a4070 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.180+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f480683c0 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.181+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6f440108d0 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.182+0000 7f6f3e7fc700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6f340a11b0 0x7f6f340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.182+0000 7f6f4ce6e700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6f340a11b0 0x7f6f340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.182+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f440ccb50 con 0x7f6f480ffba0 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.182+0000 7f6f3e7fc700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] conn(0x7f6f340aa990 0x7f6f340acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.182+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 --> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f340ad460 con 0x7f6f340aa990 2024-05-08T21:53:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.182+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f440ccff0 con 0x7f6f480ffba0 2024-05-08T21:53:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.183+0000 7f6f4de70700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] conn(0x7f6f340aa990 0x7f6f340acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.183+0000 7f6f4ce6e700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6f340a11b0 0x7f6f340a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009f50 tx=0x7f6f38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.183+0000 7f6f4de70700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] conn(0x7f6f340aa990 0x7f6f340acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.184+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== osd.0 v2:172.21.15.64:6802/4148876663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6f340ad460 con 0x7f6f340aa990 2024-05-08T21:53:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.200+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 --> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f481111b0 con 0x7f6f340aa990 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f3e7fc700 1 -- 172.21.15.64:0/175202862 <== osd.0 v2:172.21.15.64:6802/4148876663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f481111b0 con 0x7f6f340aa990 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] conn(0x7f6f340aa990 msgr2=0x7f6f340acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] conn(0x7f6f340aa990 0x7f6f340acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6f340a11b0 msgr2=0x7f6f340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6f340a11b0 0x7f6f340a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009f50 tx=0x7f6f38005fb0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f480ffba0 msgr2=0x7f6f48197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 --2- 172.21.15.64:0/175202862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f480ffba0 0x7f6f48197300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400d950 tx=0x7f6f4400dc60 comp rx=0 tx=0).stop 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4de70700 1 -- 172.21.15.64:0/175202862 reap_dead start 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 shutdown_connections 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 >> 172.21.15.64:0/175202862 conn(0x7f6f48075260 msgr2=0x7f6f480fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.201+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 shutdown_connections 2024-05-08T21:53:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.202+0000 7f6f4f8d3700 1 -- 172.21.15.64:0/175202862 wait complete. 2024-05-08T21:53:51.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:51 smithi064 ceph-mon[42500]: pgmap v234: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 -- 172.21.15.64:0/1115309823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 msgr2=0x7f39ac105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 --2- 172.21.15.64:0/1115309823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac105c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009990 tx=0x7f39a0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 -- 172.21.15.64:0/1115309823 shutdown_connections 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 --2- 172.21.15.64:0/1115309823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39ac069900 0x7f39ac069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 --2- 172.21.15.64:0/1115309823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac1062f0 0x7f39ac10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 --2- 172.21.15.64:0/1115309823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac105c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 -- 172.21.15.64:0/1115309823 >> 172.21.15.64:0/1115309823 conn(0x7f39ac076ef0 msgr2=0x7f39ac077300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 -- 172.21.15.64:0/1115309823 shutdown_connections 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.071+0000 7f39b175e700 1 -- 172.21.15.64:0/1115309823 wait complete. 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 Processor -- start 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 -- start start 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39ac069900 0x7f39ac1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac1062f0 0x7f39ac19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19c2d0 con 0x7f39ac06a2b0 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19c440 con 0x7f39ac069900 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39b175e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19c600 con 0x7f39ac1062f0 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39aa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39aa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45150/0 (socket says 172.21.15.64:45150) 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.072+0000 7f39aa7fc700 1 -- 172.21.15.64:0/1238509593 learned_addr learned my addr 172.21.15.64:0/1238509593 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aaffd700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39ac069900 0x7f39ac1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39ab7fe700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac1062f0 0x7f39ac19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aa7fc700 1 -- 172.21.15.64:0/1238509593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac1062f0 msgr2=0x7f39ac19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aa7fc700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac1062f0 0x7f39ac19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aa7fc700 1 -- 172.21.15.64:0/1238509593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39ac069900 msgr2=0x7f39ac1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aa7fc700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39ac069900 0x7f39ac1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aa7fc700 1 -- 172.21.15.64:0/1238509593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00095f0 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39aa7fc700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac197910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39a0003730 tx=0x7f39a000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a001d070 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac19c880 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a00054f0 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0004bd0 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.073+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac1a4140 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.074+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39ac111160 con 0x7f39ac06a2b0 2024-05-08T21:53:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.075+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f39a0004d30 con 0x7f39ac06a2b0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f3993fff700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f39940a11b0 0x7f39940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39a00ce170 con 0x7f39ac06a2b0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f39aaffd700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f39940a11b0 0x7f39940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f3993fff700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] conn(0x7f39940aa990 0x7f39940acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 --> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39940ad460 con 0x7f39940aa990 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a00ce610 con 0x7f39ac06a2b0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f39ab7fe700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] conn(0x7f39940aa990 0x7f39940acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.076+0000 7f39aaffd700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f39940a11b0 0x7f39940a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f399c005950 tx=0x7f399c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.077+0000 7f39ab7fe700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] conn(0x7f39940aa990 0x7f39940acdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.077+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== osd.6 v2:172.21.15.64:6818/3400846142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f39940ad460 con 0x7f39940aa990 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 --> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39ac061620 con 0x7f39940aa990 2024-05-08T21:53:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f3993fff700 1 -- 172.21.15.64:0/1238509593 <== osd.6 v2:172.21.15.64:6818/3400846142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f39ac061620 con 0x7f39940aa990 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] conn(0x7f39940aa990 msgr2=0x7f39940acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] conn(0x7f39940aa990 0x7f39940acdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f39940a11b0 msgr2=0x7f39940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f39940a11b0 0x7f39940a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f399c005950 tx=0x7f399c0058e0 comp rx=0 tx=0).stop 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 msgr2=0x7f39ac197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 --2- 172.21.15.64:0/1238509593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ac06a2b0 0x7f39ac197910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39a0003730 tx=0x7f39a000f740 comp rx=0 tx=0).stop 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39ab7fe700 1 -- 172.21.15.64:0/1238509593 reap_dead start 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 shutdown_connections 2024-05-08T21:53:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.094+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 >> 172.21.15.64:0/1238509593 conn(0x7f39ac076ef0 msgr2=0x7f39ac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.095+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 shutdown_connections 2024-05-08T21:53:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.095+0000 7f39b175e700 1 -- 172.21.15.64:0/1238509593 wait complete. 2024-05-08T21:53:51.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:51 smithi160 ceph-mon[46962]: pgmap v234: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-08T21:53:51.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:51 smithi186 ceph-mon[46619]: pgmap v234: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-08T21:53:51.711 INFO:teuthology.orchestra.run.smithi064.stdout:287762808851 2024-05-08T21:53:51.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T21:53:52.026 INFO:teuthology.orchestra.run.smithi064.stdout:34359738422 2024-05-08T21:53:52.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T21:53:52.171 INFO:teuthology.orchestra.run.smithi064.stdout:184683593761 2024-05-08T21:53:52.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T21:53:52.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 -- 172.21.15.64:0/1925395638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14100ff0 msgr2=0x7ffb14101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/1925395638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14100ff0 0x7ffb14101450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009990 tx=0x7ffb08009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 -- 172.21.15.64:0/1925395638 shutdown_connections 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/1925395638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb14107030 0x7ffb14107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/1925395638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb14101990 0x7ffb1410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/1925395638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14100ff0 0x7ffb14101450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 -- 172.21.15.64:0/1925395638 >> 172.21.15.64:0/1925395638 conn(0x7ffb14078190 msgr2=0x7ffb140785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 -- 172.21.15.64:0/1925395638 shutdown_connections 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:51.999+0000 7ffb18ad2700 1 -- 172.21.15.64:0/1925395638 wait complete. 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 Processor -- start 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 -- start start 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb14100ff0 0x7ffb141973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb14101990 0x7ffb14197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14107030 0x7ffb1419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1419c220 con 0x7ffb14107030 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1419c390 con 0x7ffb14100ff0 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb18ad2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1419c550 con 0x7ffb14101990 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.000+0000 7ffb12d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14107030 0x7ffb1419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14107030 0x7ffb1419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45198/0 (socket says 172.21.15.64:45198) 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 -- 172.21.15.64:0/3920074754 learned_addr learned my addr 172.21.15.64:0/3920074754 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb1259c700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb14100ff0 0x7ffb141973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb11d9b700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb14101990 0x7ffb14197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 -- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb14101990 msgr2=0x7ffb14197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb14101990 0x7ffb14197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 -- 172.21.15.64:0/3920074754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb14100ff0 msgr2=0x7ffb141973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb14100ff0 0x7ffb141973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 -- 172.21.15.64:0/3920074754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb080095f0 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb11d9b700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb14101990 0x7ffb14197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb12d9d700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14107030 0x7ffb1419bc10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00cca0 tx=0x7ffb0c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00aba0 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1419c830 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb0c00ad00 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0115e0 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.001+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141a40b0 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.002+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb141111b0 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.005+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ffb0c012420 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.005+0000 7ffb037fe700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffafc0a11b0 0x7ffafc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffb0c0ccf20 con 0x7ffb14107030 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb1259c700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffafc0a11b0 0x7ffafc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb037fe700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] conn(0x7ffafc0aa990 0x7ffafc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 --> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc0ad460 con 0x7ffafc0aa990 2024-05-08T21:53:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb0c0cd3c0 con 0x7ffb14107030 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb11d9b700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] conn(0x7ffafc0aa990 0x7ffafc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb1259c700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffafc0a11b0 0x7ffafc0a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb14078c20 tx=0x7ffb0400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.006+0000 7ffb11d9b700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] conn(0x7ffafc0aa990 0x7ffafc0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.007+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== osd.2 v2:172.21.15.186:6800/348681043 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ffafc0ad460 con 0x7ffafc0aa990 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.024+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 --> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb140616a0 con 0x7ffafc0aa990 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb037fe700 1 -- 172.21.15.64:0/3920074754 <== osd.2 v2:172.21.15.186:6800/348681043 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb140616a0 con 0x7ffafc0aa990 2024-05-08T21:53:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] conn(0x7ffafc0aa990 msgr2=0x7ffafc0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] conn(0x7ffafc0aa990 0x7ffafc0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffafc0a11b0 msgr2=0x7ffafc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffafc0a11b0 0x7ffafc0a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb14078c20 tx=0x7ffb0400b410 comp rx=0 tx=0).stop 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14107030 msgr2=0x7ffb1419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 --2- 172.21.15.64:0/3920074754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14107030 0x7ffb1419bc10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00cca0 tx=0x7ffb0c00a060 comp rx=0 tx=0).stop 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb12d9d700 1 -- 172.21.15.64:0/3920074754 reap_dead start 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 shutdown_connections 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.025+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 >> 172.21.15.64:0/3920074754 conn(0x7ffb14078190 msgr2=0x7ffb140ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.026+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 shutdown_connections 2024-05-08T21:53:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:52.026+0000 7ffb18ad2700 1 -- 172.21.15.64:0/3920074754 wait complete. 2024-05-08T21:53:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:52.544 INFO:teuthology.orchestra.run.smithi064.stdout:77309411376 2024-05-08T21:53:52.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 -- 172.21.15.64:0/567724621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80100890 msgr2=0x7f2a8010cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 --2- 172.21.15.64:0/567724621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80100890 0x7f2a8010cd40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b0a0 tx=0x7f2a7800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 -- 172.21.15.64:0/567724621 shutdown_connections 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 --2- 172.21.15.64:0/567724621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80100890 0x7f2a8010cd40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 --2- 172.21.15.64:0/567724621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80105f00 0x7f2a801062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 --2- 172.21.15.64:0/567724621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a800ffed0 0x7f2a80100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 -- 172.21.15.64:0/567724621 >> 172.21.15.64:0/567724621 conn(0x7f2a80075f80 msgr2=0x7f2a80076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 -- 172.21.15.64:0/567724621 shutdown_connections 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.430+0000 7f2a8714d700 1 -- 172.21.15.64:0/567724621 wait complete. 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.431+0000 7f2a8714d700 1 Processor -- start 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.431+0000 7f2a8714d700 1 -- start start 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.431+0000 7f2a8714d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a800ffed0 0x7f2a80198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a8714d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a80100890 0x7f2a801987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a8714d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 0x7f2a801a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a8714d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80199020 con 0x7f2a80105f00 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a8714d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801a3850 con 0x7f2a80100890 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a8714d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801a3990 con 0x7f2a800ffed0 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 0x7f2a801a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 0x7f2a801a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52306/0 (socket says 172.21.15.64:52306) 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 -- 172.21.15.64:0/2603053017 learned_addr learned my addr 172.21.15.64:0/2603053017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a84ee9700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a800ffed0 0x7f2a80198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a7ffff700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a80100890 0x7f2a801987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 -- 172.21.15.64:0/2603053017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a800ffed0 msgr2=0x7f2a80198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a800ffed0 0x7f2a80198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 -- 172.21.15.64:0/2603053017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a80100890 msgr2=0x7f2a801987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a80100890 0x7f2a801987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 -- 172.21.15.64:0/2603053017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78009d00 con 0x7f2a80105f00 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a856ea700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 0x7f2a801a3310 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007870 tx=0x7f2a780078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800e040 con 0x7f2a80105f00 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a801a3b90 con 0x7f2a80105f00 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a78009440 con 0x7f2a80105f00 2024-05-08T21:53:53.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.432+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7801b450 con 0x7f2a80105f00 2024-05-08T21:53:53.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.433+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a801a4050 con 0x7f2a80105f00 2024-05-08T21:53:53.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.434+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a80108420 con 0x7f2a80105f00 2024-05-08T21:53:53.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.435+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2a78004c60 con 0x7f2a80105f00 2024-05-08T21:53:53.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.436+0000 7f2a7dffb700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2a700a11b0 0x7f2a700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.436+0000 7f2a84ee9700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2a700a11b0 0x7f2a700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.436+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2a780ce100 con 0x7f2a80105f00 2024-05-08T21:53:53.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.436+0000 7f2a84ee9700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2a700a11b0 0x7f2a700a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400abd0 tx=0x7f2a7400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:53.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.440+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a78097870 con 0x7f2a80105f00 2024-05-08T21:53:53.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:53 smithi064 ceph-mon[42500]: pgmap v235: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T21:53:53.489 INFO:teuthology.orchestra.run.smithi064.stdout:154618822692 2024-05-08T21:53:53.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T21:53:53.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:53 smithi160 ceph-mon[46962]: pgmap v235: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T21:53:53.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:53 smithi186 ceph-mon[46619]: pgmap v235: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T21:53:53.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.655+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2a80068f00 con 0x7f2a80105f00 2024-05-08T21:53:53.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.655+0000 7f2a7dffb700 1 -- 172.21.15.64:0/2603053017 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a7809ae90 con 0x7f2a80105f00 2024-05-08T21:53:53.656 INFO:teuthology.orchestra.run.smithi064.stdout:103079215149 2024-05-08T21:53:53.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2a700a11b0 msgr2=0x7f2a700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2a700a11b0 0x7f2a700a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400abd0 tx=0x7f2a7400a380 comp rx=0 tx=0).stop 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 msgr2=0x7f2a801a3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 0x7f2a801a3310 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007870 tx=0x7f2a780078a0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 shutdown_connections 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f2a700a11b0 0x7f2a700a3670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a80105f00 0x7f2a801a3310 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a800ffed0 0x7f2a80198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 --2- 172.21.15.64:0/2603053017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a80100890 0x7f2a801987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 >> 172.21.15.64:0/2603053017 conn(0x7f2a80075f80 msgr2=0x7f2a800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 shutdown_connections 2024-05-08T21:53:53.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.658+0000 7f2a8714d700 1 -- 172.21.15.64:0/2603053017 wait complete. 2024-05-08T21:53:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 -- 172.21.15.64:0/3666642644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 msgr2=0x7fd63c069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 --2- 172.21.15.64:0/3666642644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c069310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd630009990 tx=0x7fd630009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 -- 172.21.15.64:0/3666642644 shutdown_connections 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 --2- 172.21.15.64:0/3666642644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd63c1039d0 0x7fd63c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 --2- 172.21.15.64:0/3666642644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd63c105ec0 0x7fd63c068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 --2- 172.21.15.64:0/3666642644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c069310 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 -- 172.21.15.64:0/3666642644 >> 172.21.15.64:0/3666642644 conn(0x7fd63c075f80 msgr2=0x7fd63c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 -- 172.21.15.64:0/3666642644 shutdown_connections 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.948+0000 7fd642075700 1 -- 172.21.15.64:0/3666642644 wait complete. 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 Processor -- start 2024-05-08T21:53:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 -- start start 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c1a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd63c1039d0 0x7fd63c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd63c105ec0 0x7fd63c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19de80 con 0x7fd63c068e90 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19dff0 con 0x7fd63c105ec0 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd642075700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19e1b0 con 0x7fd63c1039d0 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c1a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c1a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52318/0 (socket says 172.21.15.64:52318) 2024-05-08T21:53:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.949+0000 7fd63b7fe700 1 -- 172.21.15.64:0/127965727 learned_addr learned my addr 172.21.15.64:0/127965727 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63affd700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd63c1039d0 0x7fd63c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63bfff700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd63c105ec0 0x7fd63c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63b7fe700 1 -- 172.21.15.64:0/127965727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd63c1039d0 msgr2=0x7fd63c1a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63b7fe700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd63c1039d0 0x7fd63c1a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63b7fe700 1 -- 172.21.15.64:0/127965727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd63c105ec0 msgr2=0x7fd63c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63b7fe700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd63c105ec0 0x7fd63c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63b7fe700 1 -- 172.21.15.64:0/127965727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300095f0 con 0x7fd63c068e90 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd63b7fe700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c1a36d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00d980 tx=0x7fd62c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c004a30 con 0x7fd63c068e90 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c0054e0 con 0x7fd63c068e90 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c009c90 con 0x7fd63c068e90 2024-05-08T21:53:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c19e490 con 0x7fd63c068e90 2024-05-08T21:53:53.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.950+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1aa6c0 con 0x7fd63c068e90 2024-05-08T21:53:53.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.951+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c1004b0 con 0x7fd63c068e90 2024-05-08T21:53:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.956+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd62c004c20 con 0x7fd63c068e90 2024-05-08T21:53:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.956+0000 7fd638ff9700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6240a12b0 0x7fd6240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.957+0000 7fd63affd700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6240a12b0 0x7fd6240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.957+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd62c0cc8c0 con 0x7fd63c068e90 2024-05-08T21:53:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.957+0000 7fd63affd700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6240a12b0 0x7fd6240a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b5c0 tx=0x7fd630005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:53.957+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd62c096020 con 0x7fd63c068e90 2024-05-08T21:53:54.118 INFO:teuthology.orchestra.run.smithi064.stdout:317827579920 2024-05-08T21:53:54.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T21:53:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:54.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.205+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd63c19ecd0 con 0x7fd63c068e90 2024-05-08T21:53:54.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.206+0000 7fd638ff9700 1 -- 172.21.15.64:0/127965727 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd62c017020 con 0x7fd63c068e90 2024-05-08T21:53:54.206 INFO:teuthology.orchestra.run.smithi064.stdout:236223201307 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.208+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6240a12b0 msgr2=0x7fd6240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.208+0000 7fd642075700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6240a12b0 0x7fd6240a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b5c0 tx=0x7fd630005960 comp rx=0 tx=0).stop 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.208+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 msgr2=0x7fd63c1a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.208+0000 7fd642075700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c1a36d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00d980 tx=0x7fd62c00dc90 comp rx=0 tx=0).stop 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.209+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 shutdown_connections 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.209+0000 7fd642075700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6240a12b0 0x7fd6240a3770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.209+0000 7fd642075700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd63c105ec0 0x7fd63c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.209+0000 7fd642075700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c068e90 0x7fd63c1a36d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.209+0000 7fd642075700 1 --2- 172.21.15.64:0/127965727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd63c1039d0 0x7fd63c1a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.209+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 >> 172.21.15.64:0/127965727 conn(0x7fd63c075f80 msgr2=0x7fd63c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.210+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 shutdown_connections 2024-05-08T21:53:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:54.214+0000 7fd642075700 1 -- 172.21.15.64:0/127965727 wait complete. 2024-05-08T21:53:54.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:54 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2603053017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T21:53:54.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:54 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2603053017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T21:53:54.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:54 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2603053017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T21:53:54.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201305 got 236223201307 for osd.8 2024-05-08T21:53:54.829 DEBUG:teuthology.parallel:result is None 2024-05-08T21:53:55.052 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:55.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:55 smithi064 ceph-mon[42500]: pgmap v236: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-08T21:53:55.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:55 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/127965727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T21:53:55.520 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:55.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:55 smithi160 ceph-mon[46962]: pgmap v236: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-08T21:53:55.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:55 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/127965727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T21:53:55.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 -- 172.21.15.64:0/3649679452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac8107020 msgr2=0x7f6ac8107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:55.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 --2- 172.21.15.64:0/3649679452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac8107020 0x7f6ac8107400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009990 tx=0x7f6ab8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 -- 172.21.15.64:0/3649679452 shutdown_connections 2024-05-08T21:53:55.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 --2- 172.21.15.64:0/3649679452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac80692f0 0x7f6ac810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 --2- 172.21.15.64:0/3649679452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac8107020 0x7f6ac8107400 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 --2- 172.21.15.64:0/3649679452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac8068950 0x7f6ac8068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 -- 172.21.15.64:0/3649679452 >> 172.21.15.64:0/3649679452 conn(0x7f6ac8075f80 msgr2=0x7f6ac8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:55.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 -- 172.21.15.64:0/3649679452 shutdown_connections 2024-05-08T21:53:55.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.557+0000 7f6accc09700 1 -- 172.21.15.64:0/3649679452 wait complete. 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 Processor -- start 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 -- start start 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac8068950 0x7f6ac80fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 0x7f6ac81004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac8107020 0x7f6ac8100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac8101f60 con 0x7f6ac80692f0 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac819b520 con 0x7f6ac8068950 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6accc09700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac819b6e0 con 0x7f6ac8107020 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 0x7f6ac81004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 0x7f6ac81004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52330/0 (socket says 172.21.15.64:52330) 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 -- 172.21.15.64:0/316468551 learned_addr learned my addr 172.21.15.64:0/316468551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac659c700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac8068950 0x7f6ac80fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:55.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac6d9d700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac8107020 0x7f6ac8100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 -- 172.21.15.64:0/316468551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac8107020 msgr2=0x7f6ac8100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac8107020 0x7f6ac8100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 -- 172.21.15.64:0/316468551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac8068950 msgr2=0x7f6ac80fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac8068950 0x7f6ac80fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.558+0000 7f6ac5d9b700 1 -- 172.21.15.64:0/316468551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80095f0 con 0x7f6ac80692f0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.559+0000 7f6ac5d9b700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 0x7f6ac81004e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000d950 tx=0x7f6ab000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.559+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0009940 con 0x7f6ac80692f0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.559+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac819b9c0 con 0x7f6ac80692f0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.559+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ab0010460 con 0x7f6ac80692f0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.559+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000f600 con 0x7f6ac80692f0 2024-05-08T21:53:55.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.559+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac819bf10 con 0x7f6ac80692f0 2024-05-08T21:53:55.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.560+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac8109540 con 0x7f6ac80692f0 2024-05-08T21:53:55.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.561+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6ab00105d0 con 0x7f6ac80692f0 2024-05-08T21:53:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.562+0000 7f6abf7fe700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6ab40a11b0 0x7f6ab40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.562+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ab00cc800 con 0x7f6ac80692f0 2024-05-08T21:53:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.562+0000 7f6ac659c700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6ab40a11b0 0x7f6ab40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.562+0000 7f6ac659c700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6ab40a11b0 0x7f6ab40a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006010 tx=0x7f6ab800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:55.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.566+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6ab0095f20 con 0x7f6ac80692f0 2024-05-08T21:53:55.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:55 smithi186 ceph-mon[46619]: pgmap v236: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-08T21:53:55.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:55 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/127965727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T21:53:55.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215147 got 103079215149 for osd.3 2024-05-08T21:53:55.647 DEBUG:teuthology.parallel:result is None 2024-05-08T21:53:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.797+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6ac8101480 con 0x7f6ac80692f0 2024-05-08T21:53:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.798+0000 7f6abf7fe700 1 -- 172.21.15.64:0/316468551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ab0099540 con 0x7f6ac80692f0 2024-05-08T21:53:55.799 INFO:teuthology.orchestra.run.smithi064.stdout:34359738423 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6ab40a11b0 msgr2=0x7f6ab40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6ab40a11b0 0x7f6ab40a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006010 tx=0x7f6ab800b560 comp rx=0 tx=0).stop 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 msgr2=0x7f6ac81004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 0x7f6ac81004e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000d950 tx=0x7f6ab000dc60 comp rx=0 tx=0).stop 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 shutdown_connections 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6ab40a11b0 0x7f6ab40a3670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac8068950 0x7f6ac80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac8107020 0x7f6ac8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 --2- 172.21.15.64:0/316468551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac80692f0 0x7f6ac81004e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 >> 172.21.15.64:0/316468551 conn(0x7f6ac8075f80 msgr2=0x7f6ac80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 shutdown_connections 2024-05-08T21:53:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:55.800+0000 7f6accc09700 1 -- 172.21.15.64:0/316468551 wait complete. 2024-05-08T21:53:56.318 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:56 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/316468551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T21:53:56.336 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738422 got 34359738423 for osd.0 2024-05-08T21:53:56.336 DEBUG:teuthology.parallel:result is None 2024-05-08T21:53:56.377 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:56.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:56 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/316468551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T21:53:56.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:56 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/316468551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T21:53:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:57.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:57 smithi064 ceph-mon[42500]: pgmap v237: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 -- 172.21.15.64:0/1625753467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 msgr2=0x7ff81010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 --2- 172.21.15.64:0/1625753467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff81010de60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff808009990 tx=0x7ff808009ca0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 -- 172.21.15.64:0/1625753467 shutdown_connections 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 --2- 172.21.15.64:0/1625753467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff81010de60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 --2- 172.21.15.64:0/1625753467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff810107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 --2- 172.21.15.64:0/1625753467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff810068950 0x7ff810068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 -- 172.21.15.64:0/1625753467 >> 172.21.15.64:0/1625753467 conn(0x7ff810075f80 msgr2=0x7ff810076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 -- 172.21.15.64:0/1625753467 shutdown_connections 2024-05-08T21:53:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.466+0000 7ff81608a700 1 -- 172.21.15.64:0/1625753467 wait complete. 2024-05-08T21:53:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.467+0000 7ff81608a700 1 Processor -- start 2024-05-08T21:53:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.467+0000 7ff81608a700 1 -- start start 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.467+0000 7ff81608a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff810068950 0x7ff8101973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff81608a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff810197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff81608a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff81019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff81608a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81019c2e0 con 0x7ff8100692f0 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff81608a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81019c450 con 0x7ff810068950 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff81608a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81019c610 con 0x7ff810107020 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff81019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff81019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48458/0 (socket says 172.21.15.64:48458) 2024-05-08T21:53:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80ffff700 1 -- 172.21.15.64:0/1998381233 learned_addr learned my addr 172.21.15.64:0/1998381233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff810197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 -- 172.21.15.64:0/1998381233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 msgr2=0x7ff81019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff81019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 -- 172.21.15.64:0/1998381233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff810068950 msgr2=0x7ff8101973e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff810068950 0x7ff8101973e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 -- 172.21.15.64:0/1998381233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080095f0 con 0x7ff8100692f0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80ffff700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff81019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.468+0000 7ff80effd700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff810197920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800d750 tx=0x7ff7f800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.469+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004220 con 0x7ff8100692f0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.469+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f8004cf0 con 0x7ff8100692f0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.469+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800f690 con 0x7ff8100692f0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.469+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81019c8f0 con 0x7ff8100692f0 2024-05-08T21:53:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.469+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8101a4150 con 0x7ff8100692f0 2024-05-08T21:53:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.472+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff7f8004750 con 0x7ff8100692f0 2024-05-08T21:53:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.473+0000 7ff80cff9700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff7fc0a1230 0x7ff7fc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.473+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7f80cc4b0 con 0x7ff8100692f0 2024-05-08T21:53:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.474+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff810109540 con 0x7ff8100692f0 2024-05-08T21:53:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.475+0000 7ff80f7fe700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff7fc0a1230 0x7ff7fc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.476+0000 7ff80f7fe700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff7fc0a1230 0x7ff7fc0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff800007f40 tx=0x7ff800007470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:57.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.480+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff7f8095b90 con 0x7ff8100692f0 2024-05-08T21:53:57.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:57 smithi160 ceph-mon[46962]: pgmap v237: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-08T21:53:57.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:57 smithi186 ceph-mon[46619]: pgmap v237: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-08T21:53:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 -- 172.21.15.64:0/754341596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04106fe0 msgr2=0x7faa041073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 --2- 172.21.15.64:0/754341596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04106fe0 0x7faa041073c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faa000099c0 tx=0x7faa00009cd0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 -- 172.21.15.64:0/754341596 shutdown_connections 2024-05-08T21:53:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 --2- 172.21.15.64:0/754341596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa04069c10 0x7faa0410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 --2- 172.21.15.64:0/754341596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04106fe0 0x7faa041073c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 --2- 172.21.15.64:0/754341596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04069250 0x7faa040696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 -- 172.21.15.64:0/754341596 >> 172.21.15.64:0/754341596 conn(0x7faa04076e70 msgr2=0x7faa04077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 -- 172.21.15.64:0/754341596 shutdown_connections 2024-05-08T21:53:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.668+0000 7faa0c77d700 1 -- 172.21.15.64:0/754341596 wait complete. 2024-05-08T21:53:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 Processor -- start 2024-05-08T21:53:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 -- start start 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa04069250 0x7faa040fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 0x7faa041004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04106fe0 0x7faa041009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041a5760 con 0x7faa04069c10 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041a58d0 con 0x7faa04069250 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0c77d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041a5a90 con 0x7faa04106fe0 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 0x7faa041004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 0x7faa041004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52356/0 (socket says 172.21.15.64:52356) 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 -- 172.21.15.64:0/2484775292 learned_addr learned my addr 172.21.15.64:0/2484775292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0a519700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa04069250 0x7faa040fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0ad1a700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04106fe0 0x7faa041009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 -- 172.21.15.64:0/2484775292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04106fe0 msgr2=0x7faa041009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04106fe0 0x7faa041009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 -- 172.21.15.64:0/2484775292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa04069250 msgr2=0x7faa040fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa04069250 0x7faa040fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa09d18700 1 -- 172.21.15.64:0/2484775292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000095f0 con 0x7faa04069c10 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.669+0000 7faa0ad1a700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04106fe0 0x7faa041009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.670+0000 7faa09d18700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 0x7faa041004a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400e9c0 tx=0x7fa9f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.670+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40097b0 con 0x7faa04069c10 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.670+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f4004cf0 con 0x7faa04069c10 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.670+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa041a5d70 con 0x7faa04069c10 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.670+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005600 con 0x7faa04069c10 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.670+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa041a6210 con 0x7faa04069c10 2024-05-08T21:53:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.671+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa041094e0 con 0x7faa04069c10 2024-05-08T21:53:57.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.673+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa9f400cc60 con 0x7faa04069c10 2024-05-08T21:53:57.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.674+0000 7fa9fb7fe700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9f00a1230 0x7fa9f00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:53:57.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.674+0000 7faa0a519700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9f00a1230 0x7fa9f00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:53:57.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.674+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa9f4014070 con 0x7faa04069c10 2024-05-08T21:53:57.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.674+0000 7faa0a519700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9f00a1230 0x7fa9f00a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa00009fd0 tx=0x7faa00019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:53:57.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.677+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa9f40965d0 con 0x7faa04069c10 2024-05-08T21:53:57.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.702+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ff81004ec60 con 0x7ff8100692f0 2024-05-08T21:53:57.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.703+0000 7ff80cff9700 1 -- 172.21.15.64:0/1998381233 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7f8016070 con 0x7ff8100692f0 2024-05-08T21:53:57.706 INFO:teuthology.orchestra.run.smithi064.stdout:266287972375 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff7fc0a1230 msgr2=0x7ff7fc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff7fc0a1230 0x7ff7fc0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff800007f40 tx=0x7ff800007470 comp rx=0 tx=0).stop 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 msgr2=0x7ff810197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff810197920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800d750 tx=0x7ff7f800da60 comp rx=0 tx=0).stop 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 shutdown_connections 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff7fc0a1230 0x7ff7fc0a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff810068950 0x7ff8101973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff810107020 0x7ff81019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 --2- 172.21.15.64:0/1998381233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100692f0 0x7ff810197920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 >> 172.21.15.64:0/1998381233 conn(0x7ff810075f80 msgr2=0x7ff8100fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 shutdown_connections 2024-05-08T21:53:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.706+0000 7ff81608a700 1 -- 172.21.15.64:0/1998381233 wait complete. 2024-05-08T21:53:57.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.892+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faa04101440 con 0x7faa04069c10 2024-05-08T21:53:57.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.893+0000 7fa9fb7fe700 1 -- 172.21.15.64:0/2484775292 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9f4099bf0 con 0x7faa04069c10 2024-05-08T21:53:57.893 INFO:teuthology.orchestra.run.smithi064.stdout:206158430238 2024-05-08T21:53:57.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9f00a1230 msgr2=0x7fa9f00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9f00a1230 0x7fa9f00a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa00009fd0 tx=0x7faa00019040 comp rx=0 tx=0).stop 2024-05-08T21:53:57.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 msgr2=0x7faa041004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 0x7faa041004a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400e9c0 tx=0x7fa9f400ecd0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 shutdown_connections 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa9f00a1230 0x7fa9f00a36f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa04069250 0x7faa040fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa04106fe0 0x7faa041009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 --2- 172.21.15.64:0/2484775292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa04069c10 0x7faa041004a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 >> 172.21.15.64:0/2484775292 conn(0x7faa04076e70 msgr2=0x7faa040fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 shutdown_connections 2024-05-08T21:53:57.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:53:57.895+0000 7faa0c77d700 1 -- 172.21.15.64:0/2484775292 wait complete. 2024-05-08T21:53:58.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972374 got 266287972375 for osd.9 2024-05-08T21:53:58.376 DEBUG:teuthology.parallel:result is None 2024-05-08T21:53:58.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:58 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1998381233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T21:53:58.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:58 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2484775292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T21:53:58.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:58 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1998381233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T21:53:58.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:58 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2484775292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T21:53:58.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:58 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1998381233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T21:53:58.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:58 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2484775292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T21:53:59.437 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:53:59 smithi064 ceph-mon[42500]: pgmap v238: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-08T21:53:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:53:59.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:53:59 smithi160 ceph-mon[46962]: pgmap v238: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-08T21:53:59.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:53:59 smithi186 ceph-mon[46619]: pgmap v238: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-08T21:53:59.736 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430236 got 206158430238 for osd.7 2024-05-08T21:53:59.736 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:00.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/652335709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc81019b0 msgr2=0x7fcfc810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:00.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/652335709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc81019b0 0x7fcfc810dde0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000b0a0 tx=0x7fcfc000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/652335709 shutdown_connections 2024-05-08T21:54:00.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/652335709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc81019b0 0x7fcfc810dde0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/652335709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc8107030 0x7fcfc8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/652335709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc8100ff0 0x7fcfc8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/652335709 >> 172.21.15.64:0/652335709 conn(0x7fcfc8078100 msgr2=0x7fcfc8078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/652335709 shutdown_connections 2024-05-08T21:54:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.337+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/652335709 wait complete. 2024-05-08T21:54:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 Processor -- start 2024-05-08T21:54:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 -- start start 2024-05-08T21:54:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc8100ff0 0x7fcfc8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 0x7fcfc8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc8107030 0x7fcfc819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc819e500 con 0x7fcfc81019b0 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc819e670 con 0x7fcfc8107030 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfcd4c3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc819e830 con 0x7fcfc8100ff0 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfc67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 0x7fcfc8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfc67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 0x7fcfc8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52372/0 (socket says 172.21.15.64:52372) 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfc67fc700 1 -- 172.21.15.64:0/3914143535 learned_addr learned my addr 172.21.15.64:0/3914143535 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfc77fe700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc8107030 0x7fcfc819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.338+0000 7fcfc6ffd700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc8100ff0 0x7fcfc8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfc67fc700 1 -- 172.21.15.64:0/3914143535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc8100ff0 msgr2=0x7fcfc8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfc67fc700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc8100ff0 0x7fcfc8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfc67fc700 1 -- 172.21.15.64:0/3914143535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc8107030 msgr2=0x7fcfc819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfc67fc700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc8107030 0x7fcfc819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfc67fc700 1 -- 172.21.15.64:0/3914143535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc0009d00 con 0x7fcfc81019b0 2024-05-08T21:54:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfc67fc700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 0x7fcfc8199b40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000e9a0 tx=0x7fcfb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0009790 con 0x7fcfc81019b0 2024-05-08T21:54:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc819eb10 con 0x7fcfc81019b0 2024-05-08T21:54:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfb000cbd0 con 0x7fcfc81019b0 2024-05-08T21:54:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0018680 con 0x7fcfc81019b0 2024-05-08T21:54:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.339+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc81a6370 con 0x7fcfc81019b0 2024-05-08T21:54:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.340+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc8102670 con 0x7fcfc81019b0 2024-05-08T21:54:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.341+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcfb000cd40 con 0x7fcfc81019b0 2024-05-08T21:54:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.342+0000 7fcfbffff700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcfb40a1230 0x7fcfb40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.342+0000 7fcfc6ffd700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcfb40a1230 0x7fcfb40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.342+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcfb0014070 con 0x7fcfc81019b0 2024-05-08T21:54:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.342+0000 7fcfc6ffd700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcfb40a1230 0x7fcfb40a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009bf0 tx=0x7fcfb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:00.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.347+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcfb0052d60 con 0x7fcfc81019b0 2024-05-08T21:54:00.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.586+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcfc8110e80 con 0x7fcfc81019b0 2024-05-08T21:54:00.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.587+0000 7fcfbffff700 1 -- 172.21.15.64:0/3914143535 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcfb0096290 con 0x7fcfc81019b0 2024-05-08T21:54:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:77309411378 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcfb40a1230 msgr2=0x7fcfb40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcfb40a1230 0x7fcfb40a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009bf0 tx=0x7fcfb8009260 comp rx=0 tx=0).stop 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 msgr2=0x7fcfc8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 0x7fcfc8199b40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000e9a0 tx=0x7fcfb000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 shutdown_connections 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcfb40a1230 0x7fcfb40a36f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfc8107030 0x7fcfc819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc8100ff0 0x7fcfc8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 --2- 172.21.15.64:0/3914143535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfc81019b0 0x7fcfc8199b40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 >> 172.21.15.64:0/3914143535 conn(0x7fcfc8078100 msgr2=0x7fcfc810c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 shutdown_connections 2024-05-08T21:54:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.590+0000 7fcfcd4c3700 1 -- 172.21.15.64:0/3914143535 wait complete. 2024-05-08T21:54:01.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 -- 172.21.15.64:0/2292801076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 msgr2=0x7f22c810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2292801076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c810ccd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b0d0 tx=0x7f22c000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 -- 172.21.15.64:0/2292801076 shutdown_connections 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2292801076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c8106080 0x7f22c8106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2292801076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c810ccd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2292801076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c81001c0 0x7f22c8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 -- 172.21.15.64:0/2292801076 >> 172.21.15.64:0/2292801076 conn(0x7f22c8076cd0 msgr2=0x7f22c80770e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 -- 172.21.15.64:0/2292801076 shutdown_connections 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.900+0000 7f22ceb92700 1 -- 172.21.15.64:0/2292801076 wait complete. 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 Processor -- start 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 -- start start 2024-05-08T21:54:01.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c81001c0 0x7f22c81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c8199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c8106080 0x7f22c819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819e560 con 0x7f22c8100b80 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819e6d0 con 0x7f22c81001c0 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22ceb92700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819e890 con 0x7f22c8106080 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c8199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c8199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52396/0 (socket says 172.21.15.64:52396) 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22c7fff700 1 -- 172.21.15.64:0/2585668336 learned_addr learned my addr 172.21.15.64:0/2585668336 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.901+0000 7f22cd12f700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c8106080 0x7f22c819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c7fff700 1 -- 172.21.15.64:0/2585668336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c8106080 msgr2=0x7f22c819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c7fff700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c8106080 0x7f22c819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c7fff700 1 -- 172.21.15.64:0/2585668336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c81001c0 msgr2=0x7f22c81996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c7fff700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c81001c0 0x7f22c81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c7fff700 1 -- 172.21.15.64:0/2585668336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009d00 con 0x7f22c8100b80 2024-05-08T21:54:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c7fff700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c8199bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22b400e9a0 tx=0x7f22b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4009790 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b400cbd0 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4018680 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c819eb70 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.902+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c81a63d0 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.904+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c81042a0 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.906+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f22b4010430 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.906+0000 7f22c5ffb700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22b80a1230 0x7f22b80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.907+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f22b4014070 con 0x7f22c8100b80 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.907+0000 7f22cc92e700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22b80a1230 0x7f22b80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.907+0000 7f22cc92e700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22b80a1230 0x7f22b80a36f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22bc005950 tx=0x7f22bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:00.915+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22b4096690 con 0x7f22c8100b80 2024-05-08T21:54:01.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.143+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f22c8101600 con 0x7f22c8100b80 2024-05-08T21:54:01.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.143+0000 7f22c5ffb700 1 -- 172.21.15.64:0/2585668336 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f22b4099cb0 con 0x7f22c8100b80 2024-05-08T21:54:01.144 INFO:teuthology.orchestra.run.smithi064.stdout:287762808853 2024-05-08T21:54:01.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.145+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22b80a1230 msgr2=0x7f22b80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.145+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22b80a1230 0x7f22b80a36f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22bc005950 tx=0x7f22bc0058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.145+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 msgr2=0x7f22c8199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.145+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c8199bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22b400e9a0 tx=0x7f22b400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 shutdown_connections 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f22b80a1230 0x7f22b80a36f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c81001c0 0x7f22c81996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c8106080 0x7f22c819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 --2- 172.21.15.64:0/2585668336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c8100b80 0x7f22c8199bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 >> 172.21.15.64:0/2585668336 conn(0x7f22c8076cd0 msgr2=0x7f22c80697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 shutdown_connections 2024-05-08T21:54:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.146+0000 7f22ceb92700 1 -- 172.21.15.64:0/2585668336 wait complete. 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 -- 172.21.15.64:0/1596325186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 msgr2=0x7f1970107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 --2- 172.21.15.64:0/1596325186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f1970107400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1960009990 tx=0x7f1960009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 -- 172.21.15.64:0/1596325186 shutdown_connections 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 --2- 172.21.15.64:0/1596325186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19700692f0 0x7f197010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 --2- 172.21.15.64:0/1596325186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f1970107400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 --2- 172.21.15.64:0/1596325186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970068950 0x7f1970068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 -- 172.21.15.64:0/1596325186 >> 172.21.15.64:0/1596325186 conn(0x7f1970075f80 msgr2=0x7f1970076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 -- 172.21.15.64:0/1596325186 shutdown_connections 2024-05-08T21:54:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.235+0000 7f1975d29700 1 -- 172.21.15.64:0/1596325186 wait complete. 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.236+0000 7f1975d29700 1 Processor -- start 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.236+0000 7f1975d29700 1 -- start start 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970068950 0x7f19701973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19700692f0 0x7f19701978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f197019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197019c2b0 con 0x7f1970107020 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197019c420 con 0x7f19700692f0 2024-05-08T21:54:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197019c5e0 con 0x7f1970068950 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f197019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f197019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52422/0 (socket says 172.21.15.64:52422) 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 -- 172.21.15.64:0/443109822 learned_addr learned my addr 172.21.15.64:0/443109822 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196f7fe700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970068950 0x7f19701973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196effd700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19700692f0 0x7f19701978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 -- 172.21.15.64:0/443109822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970068950 msgr2=0x7f19701973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970068950 0x7f19701973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 -- 172.21.15.64:0/443109822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19700692f0 msgr2=0x7f19701978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19700692f0 0x7f19701978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 -- 172.21.15.64:0/443109822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600095f0 con 0x7f1970107020 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196ffff700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f197019bca0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1968007ee0 tx=0x7f196800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968017070 con 0x7f1970107020 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196800f040 con 0x7f1970107020 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968005580 con 0x7f1970107020 2024-05-08T21:54:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.237+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197019c8c0 con 0x7f1970107020 2024-05-08T21:54:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.238+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19701a4120 con 0x7f1970107020 2024-05-08T21:54:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.238+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1970109540 con 0x7f1970107020 2024-05-08T21:54:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.240+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f196800db20 con 0x7f1970107020 2024-05-08T21:54:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.241+0000 7f196cff9700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19580a1230 0x7f19580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.241+0000 7f196f7fe700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19580a1230 0x7f19580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.241+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19680cc670 con 0x7f1970107020 2024-05-08T21:54:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.241+0000 7f196f7fe700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19580a1230 0x7f19580a36f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f196000b5c0 tx=0x7f1960005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.248+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1968095d50 con 0x7f1970107020 2024-05-08T21:54:01.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 -- 172.21.15.64:0/3662421571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c8105f00 msgr2=0x7f16c81062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 --2- 172.21.15.64:0/3662421571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c8105f00 0x7f16c81062e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009990 tx=0x7f16b8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 -- 172.21.15.64:0/3662421571 shutdown_connections 2024-05-08T21:54:01.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 --2- 172.21.15.64:0/3662421571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16c8100890 0x7f16c810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 --2- 172.21.15.64:0/3662421571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c8105f00 0x7f16c81062e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 --2- 172.21.15.64:0/3662421571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c80ffed0 0x7f16c8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 -- 172.21.15.64:0/3662421571 >> 172.21.15.64:0/3662421571 conn(0x7f16c8075f80 msgr2=0x7f16c8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 -- 172.21.15.64:0/3662421571 shutdown_connections 2024-05-08T21:54:01.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 -- 172.21.15.64:0/3662421571 wait complete. 2024-05-08T21:54:01.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.363+0000 7f16cde53700 1 Processor -- start 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 -- start start 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 0x7f16c81973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16c8100890 0x7f16c81978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8105f00 0x7f16c819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819c2a0 con 0x7f16c80ffed0 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819c410 con 0x7f16c8100890 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16cde53700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819c5d0 con 0x7f16c8105f00 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 0x7f16c81973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 0x7f16c81973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52444/0 (socket says 172.21.15.64:52444) 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 -- 172.21.15.64:0/2318873472 learned_addr learned my addr 172.21.15.64:0/2318873472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c6ffd700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16c8100890 0x7f16c81978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c7fff700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8105f00 0x7f16c819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 -- 172.21.15.64:0/2318873472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8105f00 msgr2=0x7f16c819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8105f00 0x7f16c819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 -- 172.21.15.64:0/2318873472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16c8100890 msgr2=0x7f16c81978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16c8100890 0x7f16c81978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 -- 172.21.15.64:0/2318873472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80095f0 con 0x7f16c80ffed0 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.364+0000 7f16c77fe700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 0x7f16c81973a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009fd0 tx=0x7f16b800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.365+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b801d070 con 0x7f16c80ffed0 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.365+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c819c850 con 0x7f16c80ffed0 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.365+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c81a4110 con 0x7f16c80ffed0 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.365+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b8005050 con 0x7f16c80ffed0 2024-05-08T21:54:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.365+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8004a90 con 0x7f16c80ffed0 2024-05-08T21:54:01.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.366+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b40052f0 con 0x7f16c80ffed0 2024-05-08T21:54:01.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.367+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f16b8004bf0 con 0x7f16c80ffed0 2024-05-08T21:54:01.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.368+0000 7f16c4ff9700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b00a1230 0x7f16b00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.368+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f16b80cd8a0 con 0x7f16c80ffed0 2024-05-08T21:54:01.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.368+0000 7f16c6ffd700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b00a1230 0x7f16b00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.369+0000 7f16c6ffd700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b00a1230 0x7f16b00a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16c8198900 tx=0x7f16bc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.372+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f16b8097110 con 0x7f16c80ffed0 2024-05-08T21:54:01.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.462+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f19701986c0 con 0x7f1970107020 2024-05-08T21:54:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:01 smithi064 ceph-mon[42500]: pgmap v239: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-08T21:54:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:01 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3914143535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T21:54:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:01 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2585668336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T21:54:01.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.462+0000 7f196cff9700 1 -- 172.21.15.64:0/443109822 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1968099370 con 0x7f1970107020 2024-05-08T21:54:01.464 INFO:teuthology.orchestra.run.smithi064.stdout:133143986219 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19580a1230 msgr2=0x7f19580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19580a1230 0x7f19580a36f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f196000b5c0 tx=0x7f1960005960 comp rx=0 tx=0).stop 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 msgr2=0x7f197019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f197019bca0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1968007ee0 tx=0x7f196800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 shutdown_connections 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19580a1230 0x7f19580a36f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1970107020 0x7f197019bca0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970068950 0x7f19701973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 --2- 172.21.15.64:0/443109822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19700692f0 0x7f19701978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.465+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 >> 172.21.15.64:0/443109822 conn(0x7f1970075f80 msgr2=0x7f19700fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.466+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 shutdown_connections 2024-05-08T21:54:01.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.466+0000 7f1975d29700 1 -- 172.21.15.64:0/443109822 wait complete. 2024-05-08T21:54:01.493 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411376 got 77309411378 for osd.2 2024-05-08T21:54:01.494 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:01 smithi160 ceph-mon[46962]: pgmap v239: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-08T21:54:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:01 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3914143535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T21:54:01.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:01 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2585668336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T21:54:01.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.588+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f16b4005160 con 0x7f16c80ffed0 2024-05-08T21:54:01.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.588+0000 7f16c4ff9700 1 -- 172.21.15.64:0/2318873472 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16b809a730 con 0x7f16c80ffed0 2024-05-08T21:54:01.589 INFO:teuthology.orchestra.run.smithi064.stdout:154618822695 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b00a1230 msgr2=0x7f16b00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b00a1230 0x7f16b00a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16c8198900 tx=0x7f16bc0073d0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 msgr2=0x7f16c81973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 0x7f16c81973a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009fd0 tx=0x7f16b800f670 comp rx=0 tx=0).stop 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 shutdown_connections 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b00a1230 0x7f16b00a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8105f00 0x7f16c819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80ffed0 0x7f16c81973a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 --2- 172.21.15.64:0/2318873472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16c8100890 0x7f16c81978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 >> 172.21.15.64:0/2318873472 conn(0x7f16c8075f80 msgr2=0x7f16c80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 shutdown_connections 2024-05-08T21:54:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.590+0000 7f16cde53700 1 -- 172.21.15.64:0/2318873472 wait complete. 2024-05-08T21:54:01.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:01 smithi186 ceph-mon[46619]: pgmap v239: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-08T21:54:01.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:01 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3914143535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T21:54:01.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:01 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2585668336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T21:54:01.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 -- 172.21.15.64:0/220032634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184069270 msgr2=0x7f518410d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 --2- 172.21.15.64:0/220032634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184069270 0x7f518410d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5174009990 tx=0x7f5174009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 -- 172.21.15.64:0/220032634 shutdown_connections 2024-05-08T21:54:01.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 --2- 172.21.15.64:0/220032634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5184068950 0x7f5184068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 --2- 172.21.15.64:0/220032634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518410da80 0x7f518410fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 --2- 172.21.15.64:0/220032634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184069270 0x7f518410d540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 -- 172.21.15.64:0/220032634 >> 172.21.15.64:0/220032634 conn(0x7f5184075f80 msgr2=0x7f5184076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 -- 172.21.15.64:0/220032634 shutdown_connections 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.008+0000 7f518c24d700 1 -- 172.21.15.64:0/220032634 wait complete. 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 Processor -- start 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 -- start start 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 0x7f51841996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5184069270 0x7f5184199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f518410da80 0x7f518419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518419e5b0 con 0x7f5184068950 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518419e720 con 0x7f518410da80 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518c24d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518419e8e0 con 0x7f5184069270 2024-05-08T21:54:01.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 0x7f51841996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 0x7f51841996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52408/0 (socket says 172.21.15.64:52408) 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 -- 172.21.15.64:0/1612628379 learned_addr learned my addr 172.21.15.64:0/1612628379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f518a7ea700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f518410da80 0x7f518419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f51897e8700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5184069270 0x7f5184199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 -- 172.21.15.64:0/1612628379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5184069270 msgr2=0x7f5184199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5184069270 0x7f5184199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 -- 172.21.15.64:0/1612628379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f518410da80 msgr2=0x7f518419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f518410da80 0x7f518419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.009+0000 7f5189fe9700 1 -- 172.21.15.64:0/1612628379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740095f0 con 0x7f5184068950 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f518a7ea700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f518410da80 0x7f518419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f5189fe9700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 0x7f51841996b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9a0 tx=0x7f518000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180009790 con 0x7f5184068950 2024-05-08T21:54:01.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518419ebc0 con 0x7f5184068950 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518000cbd0 con 0x7f5184068950 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180018680 con 0x7f5184068950 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.010+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51841a6420 con 0x7f5184068950 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.011+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5184108ca0 con 0x7f5184068950 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.012+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f518000cd40 con 0x7f5184068950 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.013+0000 7f517affd700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f51700a11b0 0x7f51700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.013+0000 7f51897e8700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f51700a11b0 0x7f51700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:01.732 INFO:teuthology.orchestra.run.smithi064.stdout:55834574899 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.013+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5180014070 con 0x7f5184068950 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.013+0000 7f51897e8700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f51700a11b0 0x7f51700a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5174006010 tx=0x7f517400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.016+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5180096890 con 0x7f5184068950 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.252+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51841004b0 con 0x7f5184068950 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.252+0000 7f517affd700 1 -- 172.21.15.64:0/1612628379 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5180099eb0 con 0x7f5184068950 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f51700a11b0 msgr2=0x7f51700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f51700a11b0 0x7f51700a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5174006010 tx=0x7f517400b260 comp rx=0 tx=0).stop 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 msgr2=0x7f51841996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 0x7f51841996b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9a0 tx=0x7f518000ed60 comp rx=0 tx=0).stop 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 shutdown_connections 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f51700a11b0 0x7f51700a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f518410da80 0x7f518419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5184068950 0x7f51841996b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 --2- 172.21.15.64:0/1612628379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5184069270 0x7f5184199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:01.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 >> 172.21.15.64:0/1612628379 conn(0x7f5184075f80 msgr2=0x7f518410f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:01.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 shutdown_connections 2024-05-08T21:54:01.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:01.255+0000 7f518c24d700 1 -- 172.21.15.64:0/1612628379 wait complete. 2024-05-08T21:54:02.302 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808851 got 287762808853 for osd.10 2024-05-08T21:54:02.302 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:02.434 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1612628379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T21:54:02.434 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/443109822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T21:54:02.434 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2318873472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T21:54:02.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822692 got 154618822695 for osd.5 2024-05-08T21:54:02.452 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:02.472 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574897 got 55834574899 for osd.1 2024-05-08T21:54:02.472 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:02.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1612628379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T21:54:02.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/443109822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T21:54:02.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2318873472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 -- 172.21.15.64:0/1296760653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c1001f0 msgr2=0x7fc53c100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 --2- 172.21.15.64:0/1296760653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c1001f0 0x7fc53c100670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc528009990 tx=0x7fc528009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 -- 172.21.15.64:0/1296760653 shutdown_connections 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 --2- 172.21.15.64:0/1296760653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c100bb0 0x7fc53c10cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 --2- 172.21.15.64:0/1296760653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc53c105f80 0x7fc53c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 --2- 172.21.15.64:0/1296760653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c1001f0 0x7fc53c100670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 -- 172.21.15.64:0/1296760653 >> 172.21.15.64:0/1296760653 conn(0x7fc53c076c50 msgr2=0x7fc53c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 -- 172.21.15.64:0/1296760653 shutdown_connections 2024-05-08T21:54:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.601+0000 7fc542f08700 1 -- 172.21.15.64:0/1296760653 wait complete. 2024-05-08T21:54:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 Processor -- start 2024-05-08T21:54:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 -- start start 2024-05-08T21:54:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c1001f0 0x7fc53c196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 0x7fc53c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc53c105f80 0x7fc53c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c19b210 con 0x7fc53c100bb0 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c19b380 con 0x7fc53c1001f0 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc542f08700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c19b540 con 0x7fc53c105f80 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc53bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 0x7fc53c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc53bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 0x7fc53c196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50410/0 (socket says 172.21.15.64:50410) 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.602+0000 7fc53bfff700 1 -- 172.21.15.64:0/3239900074 learned_addr learned my addr 172.21.15.64:0/3239900074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc540ca4700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c1001f0 0x7fc53c196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc53bfff700 1 -- 172.21.15.64:0/3239900074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc53c105f80 msgr2=0x7fc53c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc53bfff700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc53c105f80 0x7fc53c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc53bfff700 1 -- 172.21.15.64:0/3239900074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c1001f0 msgr2=0x7fc53c196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc53bfff700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c1001f0 0x7fc53c196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc53bfff700 1 -- 172.21.15.64:0/3239900074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280095f0 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc53bfff700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 0x7fc53c196850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5280098e0 tx=0x7fc52800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52801d070 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c19b7c0 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc528022470 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528017650 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.603+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c1a3080 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.604+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53c108410 con 0x7fc53c100bb0 2024-05-08T21:54:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.606+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc5280225e0 con 0x7fc53c100bb0 2024-05-08T21:54:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.607+0000 7fc539ffb700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc52c0a11b0 0x7fc52c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.607+0000 7fc540ca4700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc52c0a11b0 0x7fc52c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.607+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5280cdc80 con 0x7fc53c100bb0 2024-05-08T21:54:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.607+0000 7fc540ca4700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc52c0a11b0 0x7fc52c0a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc530005950 tx=0x7fc53000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:02.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.610+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc5280974f0 con 0x7fc53c100bb0 2024-05-08T21:54:02.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1612628379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T21:54:02.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/443109822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T21:54:02.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2318873472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T21:54:02.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.820+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc53c068e90 con 0x7fc53c100bb0 2024-05-08T21:54:02.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.820+0000 7fc539ffb700 1 -- 172.21.15.64:0/3239900074 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc52809ab10 con 0x7fc53c100bb0 2024-05-08T21:54:02.821 INFO:teuthology.orchestra.run.smithi064.stdout:317827579922 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.822+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc52c0a11b0 msgr2=0x7fc52c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.822+0000 7fc542f08700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc52c0a11b0 0x7fc52c0a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc530005950 tx=0x7fc53000b410 comp rx=0 tx=0).stop 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.822+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 msgr2=0x7fc53c196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.822+0000 7fc542f08700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 0x7fc53c196850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5280098e0 tx=0x7fc52800f670 comp rx=0 tx=0).stop 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 shutdown_connections 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc52c0a11b0 0x7fc52c0a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c1001f0 0x7fc53c196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc53c105f80 0x7fc53c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 --2- 172.21.15.64:0/3239900074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53c100bb0 0x7fc53c196850 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:02.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 >> 172.21.15.64:0/3239900074 conn(0x7fc53c076c50 msgr2=0x7fc53c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:02.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 shutdown_connections 2024-05-08T21:54:02.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:02.823+0000 7fc542f08700 1 -- 172.21.15.64:0/3239900074 wait complete. 2024-05-08T21:54:02.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986216 got 133143986219 for osd.4 2024-05-08T21:54:02.884 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:03.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:03 smithi064 ceph-mon[42500]: pgmap v240: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T21:54:03.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:03 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3239900074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T21:54:03.304 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579920 got 317827579922 for osd.11 2024-05-08T21:54:03.304 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 -- 172.21.15.64:0/3324548990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c105f00 msgr2=0x7f801c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3324548990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c105f00 0x7f801c1062e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8010009990 tx=0x7f8010009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 -- 172.21.15.64:0/3324548990 shutdown_connections 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3324548990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f801c100890 0x7f801c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3324548990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c105f00 0x7f801c1062e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3324548990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f801c0ffed0 0x7f801c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 -- 172.21.15.64:0/3324548990 >> 172.21.15.64:0/3324548990 conn(0x7f801c075f80 msgr2=0x7f801c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:03.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 -- 172.21.15.64:0/3324548990 shutdown_connections 2024-05-08T21:54:03.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 -- 172.21.15.64:0/3324548990 wait complete. 2024-05-08T21:54:03.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 Processor -- start 2024-05-08T21:54:03.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.427+0000 7f8022b8b700 1 -- start start 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 0x7f801c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f801c100890 0x7f801c1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f801c105f00 0x7f801c1025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1a57c0 con 0x7f801c0ffed0 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1a5930 con 0x7f801c100890 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1a5af0 con 0x7f801c105f00 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 0x7f801c101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 0x7f801c101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50432/0 (socket says 172.21.15.64:50432) 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 -- 172.21.15.64:0/3023748175 learned_addr learned my addr 172.21.15.64:0/3023748175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f801bfff700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f801c100890 0x7f801c1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8021128700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f801c105f00 0x7f801c1025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 -- 172.21.15.64:0/3023748175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f801c105f00 msgr2=0x7f801c1025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f801c105f00 0x7f801c1025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 -- 172.21.15.64:0/3023748175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f801c100890 msgr2=0x7f801c1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f801c100890 0x7f801c1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 -- 172.21.15.64:0/3023748175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100095f0 con 0x7f801c0ffed0 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8020927700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 0x7f801c101b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80100098e0 tx=0x7f801000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801001d070 con 0x7f801c0ffed0 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8010022470 con 0x7f801c0ffed0 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c1a5d70 con 0x7f801c0ffed0 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.428+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010017650 con 0x7f801c0ffed0 2024-05-08T21:54:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.429+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c1a6190 con 0x7f801c0ffed0 2024-05-08T21:54:03.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.429+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c1015b0 con 0x7f801c0ffed0 2024-05-08T21:54:03.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.431+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f80100225e0 con 0x7f801c0ffed0 2024-05-08T21:54:03.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.431+0000 7f8019ffb700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f800c0a1230 0x7f800c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:03.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.432+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f80100cdbc0 con 0x7f801c0ffed0 2024-05-08T21:54:03.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.432+0000 7f801bfff700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f800c0a1230 0x7f800c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:03.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.432+0000 7f801bfff700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f800c0a1230 0x7f800c0a36f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f801c1032a0 tx=0x7f800800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:03.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.435+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8010097430 con 0x7f801c0ffed0 2024-05-08T21:54:03.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:03 smithi160 ceph-mon[46962]: pgmap v240: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T21:54:03.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:03 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3239900074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T21:54:03.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:03 smithi186 ceph-mon[46619]: pgmap v240: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T21:54:03.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:03 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3239900074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T21:54:03.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.642+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f801c10fdc0 con 0x7f801c0ffed0 2024-05-08T21:54:03.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.642+0000 7f8019ffb700 1 -- 172.21.15.64:0/3023748175 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f801009aa50 con 0x7f801c0ffed0 2024-05-08T21:54:03.642 INFO:teuthology.orchestra.run.smithi064.stdout:184683593763 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f800c0a1230 msgr2=0x7f800c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f800c0a1230 0x7f800c0a36f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f801c1032a0 tx=0x7f800800b410 comp rx=0 tx=0).stop 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 msgr2=0x7f801c101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 0x7f801c101b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80100098e0 tx=0x7f801000f670 comp rx=0 tx=0).stop 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 shutdown_connections 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f800c0a1230 0x7f800c0a36f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f801c105f00 0x7f801c1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f801c0ffed0 0x7f801c101b70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 --2- 172.21.15.64:0/3023748175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f801c100890 0x7f801c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.644+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 >> 172.21.15.64:0/3023748175 conn(0x7f801c075f80 msgr2=0x7f801c068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.645+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 shutdown_connections 2024-05-08T21:54:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:03.645+0000 7f8022b8b700 1 -- 172.21.15.64:0/3023748175 wait complete. 2024-05-08T21:54:04.052 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593761 got 184683593763 for osd.6 2024-05-08T21:54:04.052 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:04.052 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T21:54:04.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T21:54:04.286 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:04.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:04 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3023748175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T21:54:04.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:04 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3023748175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T21:54:04.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:04 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3023748175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T21:54:05.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 -- 172.21.15.64:0/3019889906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 msgr2=0x7f0b801062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:05.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/3019889906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b801062e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009990 tx=0x7f0b70009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 -- 172.21.15.64:0/3019889906 shutdown_connections 2024-05-08T21:54:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/3019889906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b80100890 0x7f0b8010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/3019889906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b801062e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/3019889906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b800ffed0 0x7f0b80100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 -- 172.21.15.64:0/3019889906 >> 172.21.15.64:0/3019889906 conn(0x7f0b80075f80 msgr2=0x7f0b80076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 -- 172.21.15.64:0/3019889906 shutdown_connections 2024-05-08T21:54:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 -- 172.21.15.64:0/3019889906 wait complete. 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.302+0000 7f0b84a1f700 1 Processor -- start 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 -- start start 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b800ffed0 0x7f0b801974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b80100890 0x7f0b801979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b8019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019c3a0 con 0x7f0b80105f00 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019c510 con 0x7f0b80100890 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b84a1f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019c6d0 con 0x7f0b800ffed0 2024-05-08T21:54:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b8019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b8019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50440/0 (socket says 172.21.15.64:50440) 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b80100890 0x7f0b801979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 -- 172.21.15.64:0/2897803144 learned_addr learned my addr 172.21.15.64:0/2897803144 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7e59c700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b800ffed0 0x7f0b801974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 -- 172.21.15.64:0/2897803144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b800ffed0 msgr2=0x7f0b801974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b800ffed0 0x7f0b801974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 -- 172.21.15.64:0/2897803144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b80100890 msgr2=0x7f0b801979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b80100890 0x7f0b801979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 -- 172.21.15.64:0/2897803144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700095f0 con 0x7f0b80105f00 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7dd9b700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b80100890 0x7f0b801979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.303+0000 7f0b7ed9d700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b8019bd90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800ada0 tx=0x7f0b7800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.304+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800f070 con 0x7f0b80105f00 2024-05-08T21:54:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.304+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8019c9b0 con 0x7f0b80105f00 2024-05-08T21:54:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.304+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b78007b30 con 0x7f0b80105f00 2024-05-08T21:54:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.304+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801a4210 con 0x7f0b80105f00 2024-05-08T21:54:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.304+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78005280 con 0x7f0b80105f00 2024-05-08T21:54:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.305+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b600052f0 con 0x7f0b80105f00 2024-05-08T21:54:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.306+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0b780054e0 con 0x7f0b80105f00 2024-05-08T21:54:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.306+0000 7f0b6f7fe700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0b680a1230 0x7f0b680a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.306+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0b780cbc70 con 0x7f0b80105f00 2024-05-08T21:54:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.307+0000 7f0b7e59c700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0b680a1230 0x7f0b680a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.307+0000 7f0b7e59c700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0b680a1230 0x7f0b680a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000b2e0 tx=0x7f0b70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.311+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0b780954e0 con 0x7f0b80105f00 2024-05-08T21:54:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:05 smithi064 ceph-mon[42500]: pgmap v241: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-08T21:54:05.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.512+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b60000bc0 con 0x7f0b680a1230 2024-05-08T21:54:05.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.522+0000 7f0b6f7fe700 1 -- 172.21.15.64:0/2897803144 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340464 (secure 0 0 0) 0x7f0b60000bc0 con 0x7f0b680a1230 2024-05-08T21:54:05.529 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0b680a1230 msgr2=0x7f0b680a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0b680a1230 0x7f0b680a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000b2e0 tx=0x7f0b70005fd0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 msgr2=0x7f0b8019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b8019bd90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800ada0 tx=0x7f0b7800c330 comp rx=0 tx=0).stop 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 shutdown_connections 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0b680a1230 0x7f0b680a36f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b80105f00 0x7f0b8019bd90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b800ffed0 0x7f0b801974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 --2- 172.21.15.64:0/2897803144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b80100890 0x7f0b801979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:05.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.525+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 >> 172.21.15.64:0/2897803144 conn(0x7f0b80075f80 msgr2=0x7f0b80068b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:05.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.526+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 shutdown_connections 2024-05-08T21:54:05.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:05.526+0000 7f0b84a1f700 1 -- 172.21.15.64:0/2897803144 wait complete. 2024-05-08T21:54:05.533 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-08T21:54:05.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:05 smithi160 ceph-mon[46962]: pgmap v241: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-08T21:54:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:05 smithi186 ceph-mon[46619]: pgmap v241: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-08T21:54:05.902 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-05-08T21:54:04.103409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659011,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":499,"num_read_kb":371,"num_write":415,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356912,"kb_used_data":12240,"kb_used_omap":18,"kb_used_meta":344557,"kb_avail":1124584912,"statfs":{"total":1151940427776,"available":1151574949888,"internally_reserved":0,"allocated":12533760,"data_stored":6885600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":352826693},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":58,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002816"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703161+0000","last_change":"2024-05-08T21:53:23.579256+0000","last_active":"2024-05-08T21:53:23.703161+0000","last_peered":"2024-05-08T21:53:23.703161+0000","last_clean":"2024-05-08T21:53:23.703161+0000","last_became_active":"2024-05-08T21:53:23.578867+0000","last_became_peered":"2024-05-08T21:53:23.578867+0000","last_unstale":"2024-05-08T21:53:23.703161+0000","last_undegraded":"2024-05-08T21:53:23.703161+0000","last_fullsized":"2024-05-08T21:53:23.703161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:07.123241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780009+0000","last_change":"2024-05-08T21:53:23.575229+0000","last_active":"2024-05-08T21:53:23.780009+0000","last_peered":"2024-05-08T21:53:23.780009+0000","last_clean":"2024-05-08T21:53:23.780009+0000","last_became_active":"2024-05-08T21:53:23.574933+0000","last_became_peered":"2024-05-08T21:53:23.574933+0000","last_unstale":"2024-05-08T21:53:23.780009+0000","last_undegraded":"2024-05-08T21:53:23.780009+0000","last_fullsized":"2024-05-08T21:53:23.780009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:04.592572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779970+0000","last_change":"2024-05-08T21:53:23.587183+0000","last_active":"2024-05-08T21:53:23.779970+0000","last_peered":"2024-05-08T21:53:23.779970+0000","last_clean":"2024-05-08T21:53:23.779970+0000","last_became_active":"2024-05-08T21:53:23.586861+0000","last_became_peered":"2024-05-08T21:53:23.586861+0000","last_unstale":"2024-05-08T21:53:23.779970+0000","last_undegraded":"2024-05-08T21:53:23.779970+0000","last_fullsized":"2024-05-08T21:53:23.779970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:48.878049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.587773+0000","last_change":"2024-05-08T21:53:23.587097+0000","last_active":"2024-05-08T21:53:23.587773+0000","last_peered":"2024-05-08T21:53:23.587773+0000","last_clean":"2024-05-08T21:53:23.587773+0000","last_became_active":"2024-05-08T21:53:23.586756+0000","last_became_peered":"2024-05-08T21:53:23.586756+0000","last_unstale":"2024-05-08T21:53:23.587773+0000","last_undegraded":"2024-05-08T21:53:23.587773+0000","last_fullsized":"2024-05-08T21:53:23.587773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:38.938727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566720+0000","last_change":"2024-05-08T21:53:23.581157+0000","last_active":"2024-05-08T21:53:24.566720+0000","last_peered":"2024-05-08T21:53:24.566720+0000","last_clean":"2024-05-08T21:53:24.566720+0000","last_became_active":"2024-05-08T21:53:23.580803+0000","last_became_peered":"2024-05-08T21:53:23.580803+0000","last_unstale":"2024-05-08T21:53:24.566720+0000","last_undegraded":"2024-05-08T21:53:24.566720+0000","last_fullsized":"2024-05-08T21:53:24.566720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:13:59.878096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113677+0000","last_change":"2024-05-08T21:53:23.583093+0000","last_active":"2024-05-08T21:53:25.113677+0000","last_peered":"2024-05-08T21:53:25.113677+0000","last_clean":"2024-05-08T21:53:25.113677+0000","last_became_active":"2024-05-08T21:53:23.582798+0000","last_became_peered":"2024-05-08T21:53:23.582798+0000","last_unstale":"2024-05-08T21:53:25.113677+0000","last_undegraded":"2024-05-08T21:53:25.113677+0000","last_fullsized":"2024-05-08T21:53:25.113677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:53.682817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888997+0000","last_change":"2024-05-08T21:53:23.586322+0000","last_active":"2024-05-08T21:53:25.888997+0000","last_peered":"2024-05-08T21:53:25.888997+0000","last_clean":"2024-05-08T21:53:25.888997+0000","last_became_active":"2024-05-08T21:53:23.585938+0000","last_became_peered":"2024-05-08T21:53:23.585938+0000","last_unstale":"2024-05-08T21:53:25.888997+0000","last_undegraded":"2024-05-08T21:53:25.888997+0000","last_fullsized":"2024-05-08T21:53:25.888997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:22.744048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566197+0000","last_change":"2024-05-08T21:53:23.580496+0000","last_active":"2024-05-08T21:53:24.566197+0000","last_peered":"2024-05-08T21:53:24.566197+0000","last_clean":"2024-05-08T21:53:24.566197+0000","last_became_active":"2024-05-08T21:53:23.580154+0000","last_became_peered":"2024-05-08T21:53:23.580154+0000","last_unstale":"2024-05-08T21:53:24.566197+0000","last_undegraded":"2024-05-08T21:53:24.566197+0000","last_fullsized":"2024-05-08T21:53:24.566197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:50.713758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568936+0000","last_change":"2024-05-08T21:53:23.580549+0000","last_active":"2024-05-08T21:53:24.568936+0000","last_peered":"2024-05-08T21:53:24.568936+0000","last_clean":"2024-05-08T21:53:24.568936+0000","last_became_active":"2024-05-08T21:53:23.579962+0000","last_became_peered":"2024-05-08T21:53:23.579962+0000","last_unstale":"2024-05-08T21:53:24.568936+0000","last_undegraded":"2024-05-08T21:53:24.568936+0000","last_fullsized":"2024-05-08T21:53:24.568936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:52.010295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113920+0000","last_change":"2024-05-08T21:53:23.589640+0000","last_active":"2024-05-08T21:53:25.113920+0000","last_peered":"2024-05-08T21:53:25.113920+0000","last_clean":"2024-05-08T21:53:25.113920+0000","last_became_active":"2024-05-08T21:53:23.589347+0000","last_became_peered":"2024-05-08T21:53:23.589347+0000","last_unstale":"2024-05-08T21:53:25.113920+0000","last_undegraded":"2024-05-08T21:53:25.113920+0000","last_fullsized":"2024-05-08T21:53:25.113920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:40.346414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889087+0000","last_change":"2024-05-08T21:53:23.586430+0000","last_active":"2024-05-08T21:53:25.889087+0000","last_peered":"2024-05-08T21:53:25.889087+0000","last_clean":"2024-05-08T21:53:25.889087+0000","last_became_active":"2024-05-08T21:53:23.585986+0000","last_became_peered":"2024-05-08T21:53:23.585986+0000","last_unstale":"2024-05-08T21:53:25.889087+0000","last_undegraded":"2024-05-08T21:53:25.889087+0000","last_fullsized":"2024-05-08T21:53:25.889087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:09.970481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.586950+0000","last_change":"2024-05-08T21:53:23.585441+0000","last_active":"2024-05-08T21:53:23.586950+0000","last_peered":"2024-05-08T21:53:23.586950+0000","last_clean":"2024-05-08T21:53:23.586950+0000","last_became_active":"2024-05-08T21:53:23.584809+0000","last_became_peered":"2024-05-08T21:53:23.584809+0000","last_unstale":"2024-05-08T21:53:23.586950+0000","last_undegraded":"2024-05-08T21:53:23.586950+0000","last_fullsized":"2024-05-08T21:53:23.586950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:31.599185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806575+0000","last_change":"2024-05-08T21:53:23.575757+0000","last_active":"2024-05-08T21:53:23.806575+0000","last_peered":"2024-05-08T21:53:23.806575+0000","last_clean":"2024-05-08T21:53:23.806575+0000","last_became_active":"2024-05-08T21:53:23.575376+0000","last_became_peered":"2024-05-08T21:53:23.575376+0000","last_unstale":"2024-05-08T21:53:23.806575+0000","last_undegraded":"2024-05-08T21:53:23.806575+0000","last_fullsized":"2024-05-08T21:53:23.806575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.851616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779926+0000","last_change":"2024-05-08T21:53:23.573225+0000","last_active":"2024-05-08T21:53:23.779926+0000","last_peered":"2024-05-08T21:53:23.779926+0000","last_clean":"2024-05-08T21:53:23.779926+0000","last_became_active":"2024-05-08T21:53:23.572784+0000","last_became_peered":"2024-05-08T21:53:23.572784+0000","last_unstale":"2024-05-08T21:53:23.779926+0000","last_undegraded":"2024-05-08T21:53:23.779926+0000","last_fullsized":"2024-05-08T21:53:23.779926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:11.963959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779898+0000","last_change":"2024-05-08T21:53:23.583961+0000","last_active":"2024-05-08T21:53:23.779898+0000","last_peered":"2024-05-08T21:53:23.779898+0000","last_clean":"2024-05-08T21:53:23.779898+0000","last_became_active":"2024-05-08T21:53:23.583532+0000","last_became_peered":"2024-05-08T21:53:23.583532+0000","last_unstale":"2024-05-08T21:53:23.779898+0000","last_undegraded":"2024-05-08T21:53:23.779898+0000","last_fullsized":"2024-05-08T21:53:23.779898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:15:04.318298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568388+0000","last_change":"2024-05-08T21:53:23.577361+0000","last_active":"2024-05-08T21:53:24.568388+0000","last_peered":"2024-05-08T21:53:24.568388+0000","last_clean":"2024-05-08T21:53:24.568388+0000","last_became_active":"2024-05-08T21:53:23.577003+0000","last_became_peered":"2024-05-08T21:53:23.577003+0000","last_unstale":"2024-05-08T21:53:24.568388+0000","last_undegraded":"2024-05-08T21:53:24.568388+0000","last_fullsized":"2024-05-08T21:53:24.568388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:14.617211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703171+0000","last_change":"2024-05-08T21:53:23.579881+0000","last_active":"2024-05-08T21:53:23.703171+0000","last_peered":"2024-05-08T21:53:23.703171+0000","last_clean":"2024-05-08T21:53:23.703171+0000","last_became_active":"2024-05-08T21:53:23.579569+0000","last_became_peered":"2024-05-08T21:53:23.579569+0000","last_unstale":"2024-05-08T21:53:23.703171+0000","last_undegraded":"2024-05-08T21:53:23.703171+0000","last_fullsized":"2024-05-08T21:53:23.703171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:38.210018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.591923+0000","last_change":"2024-05-08T21:53:23.573027+0000","last_active":"2024-05-08T21:53:26.591923+0000","last_peered":"2024-05-08T21:53:26.591923+0000","last_clean":"2024-05-08T21:53:26.591923+0000","last_became_active":"2024-05-08T21:53:23.572709+0000","last_became_peered":"2024-05-08T21:53:23.572709+0000","last_unstale":"2024-05-08T21:53:26.591923+0000","last_undegraded":"2024-05-08T21:53:26.591923+0000","last_fullsized":"2024-05-08T21:53:26.591923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:36.555513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.588568+0000","last_change":"2024-05-08T21:53:23.586364+0000","last_active":"2024-05-08T21:53:23.588568+0000","last_peered":"2024-05-08T21:53:23.588568+0000","last_clean":"2024-05-08T21:53:23.588568+0000","last_became_active":"2024-05-08T21:53:23.585420+0000","last_became_peered":"2024-05-08T21:53:23.585420+0000","last_unstale":"2024-05-08T21:53:23.588568+0000","last_undegraded":"2024-05-08T21:53:23.588568+0000","last_fullsized":"2024-05-08T21:53:23.588568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:01.545821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:01.020694+0000","last_change":"2024-05-08T21:53:23.148809+0000","last_active":"2024-05-08T21:54:01.020694+0000","last_peered":"2024-05-08T21:54:01.020694+0000","last_clean":"2024-05-08T21:54:01.020694+0000","last_became_active":"2024-05-08T21:53:23.148532+0000","last_became_peered":"2024-05-08T21:53:23.148532+0000","last_unstale":"2024-05-08T21:54:01.020694+0000","last_undegraded":"2024-05-08T21:54:01.020694+0000","last_fullsized":"2024-05-08T21:54:01.020694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:48.209778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566739+0000","last_change":"2024-05-08T21:53:23.584120+0000","last_active":"2024-05-08T21:53:24.566739+0000","last_peered":"2024-05-08T21:53:24.566739+0000","last_clean":"2024-05-08T21:53:24.566739+0000","last_became_active":"2024-05-08T21:53:23.583804+0000","last_became_peered":"2024-05-08T21:53:23.583804+0000","last_unstale":"2024-05-08T21:53:24.566739+0000","last_undegraded":"2024-05-08T21:53:24.566739+0000","last_fullsized":"2024-05-08T21:53:24.566739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:10.669322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.196151+0000","last_change":"2024-05-08T21:53:23.144732+0000","last_active":"2024-05-08T21:53:26.196151+0000","last_peered":"2024-05-08T21:53:26.196151+0000","last_clean":"2024-05-08T21:53:26.196151+0000","last_became_active":"2024-05-08T21:53:23.144405+0000","last_became_peered":"2024-05-08T21:53:23.144405+0000","last_unstale":"2024-05-08T21:53:26.196151+0000","last_undegraded":"2024-05-08T21:53:26.196151+0000","last_fullsized":"2024-05-08T21:53:26.196151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:46.356499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780046+0000","last_change":"2024-05-08T21:53:23.578621+0000","last_active":"2024-05-08T21:53:23.780046+0000","last_peered":"2024-05-08T21:53:23.780046+0000","last_clean":"2024-05-08T21:53:23.780046+0000","last_became_active":"2024-05-08T21:53:23.578331+0000","last_became_peered":"2024-05-08T21:53:23.578331+0000","last_unstale":"2024-05-08T21:53:23.780046+0000","last_undegraded":"2024-05-08T21:53:23.780046+0000","last_fullsized":"2024-05-08T21:53:23.780046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:27.409751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.969142+0000","last_change":"2024-05-08T21:53:23.145398+0000","last_active":"2024-05-08T21:53:29.969142+0000","last_peered":"2024-05-08T21:53:29.969142+0000","last_clean":"2024-05-08T21:53:29.969142+0000","last_became_active":"2024-05-08T21:53:23.144997+0000","last_became_peered":"2024-05-08T21:53:23.144997+0000","last_unstale":"2024-05-08T21:53:29.969142+0000","last_undegraded":"2024-05-08T21:53:29.969142+0000","last_fullsized":"2024-05-08T21:53:29.969142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.116791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568566+0000","last_change":"2024-05-08T21:53:23.573118+0000","last_active":"2024-05-08T21:53:24.568566+0000","last_peered":"2024-05-08T21:53:24.568566+0000","last_clean":"2024-05-08T21:53:24.568566+0000","last_became_active":"2024-05-08T21:53:23.572639+0000","last_became_peered":"2024-05-08T21:53:23.572639+0000","last_unstale":"2024-05-08T21:53:24.568566+0000","last_undegraded":"2024-05-08T21:53:24.568566+0000","last_fullsized":"2024-05-08T21:53:24.568566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.786457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.964190+0000","last_change":"2024-05-08T21:53:23.144706+0000","last_active":"2024-05-08T21:53:28.964190+0000","last_peered":"2024-05-08T21:53:28.964190+0000","last_clean":"2024-05-08T21:53:28.964190+0000","last_became_active":"2024-05-08T21:53:23.144380+0000","last_became_peered":"2024-05-08T21:53:23.144380+0000","last_unstale":"2024-05-08T21:53:28.964190+0000","last_undegraded":"2024-05-08T21:53:28.964190+0000","last_fullsized":"2024-05-08T21:53:28.964190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:05.594755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832631+0000","last_change":"2024-05-08T21:53:23.573836+0000","last_active":"2024-05-08T21:53:24.832631+0000","last_peered":"2024-05-08T21:53:24.832631+0000","last_clean":"2024-05-08T21:53:24.832631+0000","last_became_active":"2024-05-08T21:53:23.572323+0000","last_became_peered":"2024-05-08T21:53:23.572323+0000","last_unstale":"2024-05-08T21:53:24.832631+0000","last_undegraded":"2024-05-08T21:53:24.832631+0000","last_fullsized":"2024-05-08T21:53:24.832631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:34.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":53,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:57.994576+0000","last_change":"2024-05-08T21:53:23.147919+0000","last_active":"2024-05-08T21:53:57.994576+0000","last_peered":"2024-05-08T21:53:57.994576+0000","last_clean":"2024-05-08T21:53:57.994576+0000","last_became_active":"2024-05-08T21:53:23.147473+0000","last_became_peered":"2024-05-08T21:53:23.147473+0000","last_unstale":"2024-05-08T21:53:57.994576+0000","last_undegraded":"2024-05-08T21:53:57.994576+0000","last_fullsized":"2024-05-08T21:53:57.994576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:07.158398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566928+0000","last_change":"2024-05-08T21:53:23.576219+0000","last_active":"2024-05-08T21:53:24.566928+0000","last_peered":"2024-05-08T21:53:24.566928+0000","last_clean":"2024-05-08T21:53:24.566928+0000","last_became_active":"2024-05-08T21:53:23.575779+0000","last_became_peered":"2024-05-08T21:53:23.575779+0000","last_unstale":"2024-05-08T21:53:24.566928+0000","last_undegraded":"2024-05-08T21:53:24.566928+0000","last_fullsized":"2024-05-08T21:53:24.566928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:19.813711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953935+0000","last_change":"2024-05-08T21:53:23.152159+0000","last_active":"2024-05-08T21:53:27.953935+0000","last_peered":"2024-05-08T21:53:27.953935+0000","last_clean":"2024-05-08T21:53:27.953935+0000","last_became_active":"2024-05-08T21:53:23.151816+0000","last_became_peered":"2024-05-08T21:53:23.151816+0000","last_unstale":"2024-05-08T21:53:27.953935+0000","last_undegraded":"2024-05-08T21:53:27.953935+0000","last_fullsized":"2024-05-08T21:53:27.953935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.995097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832701+0000","last_change":"2024-05-08T21:53:23.575509+0000","last_active":"2024-05-08T21:53:24.832701+0000","last_peered":"2024-05-08T21:53:24.832701+0000","last_clean":"2024-05-08T21:53:24.832701+0000","last_became_active":"2024-05-08T21:53:23.574966+0000","last_became_peered":"2024-05-08T21:53:23.574966+0000","last_unstale":"2024-05-08T21:53:24.832701+0000","last_undegraded":"2024-05-08T21:53:24.832701+0000","last_fullsized":"2024-05-08T21:53:24.832701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:50.075249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.601006+0000","last_change":"2024-05-08T21:53:23.143151+0000","last_active":"2024-05-08T21:53:27.601006+0000","last_peered":"2024-05-08T21:53:27.601006+0000","last_clean":"2024-05-08T21:53:27.601006+0000","last_became_active":"2024-05-08T21:53:23.142829+0000","last_became_peered":"2024-05-08T21:53:23.142829+0000","last_unstale":"2024-05-08T21:53:27.601006+0000","last_undegraded":"2024-05-08T21:53:27.601006+0000","last_fullsized":"2024-05-08T21:53:27.601006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:49.372293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566098+0000","last_change":"2024-05-08T21:53:23.573587+0000","last_active":"2024-05-08T21:53:24.566098+0000","last_peered":"2024-05-08T21:53:24.566098+0000","last_clean":"2024-05-08T21:53:24.566098+0000","last_became_active":"2024-05-08T21:53:23.573329+0000","last_became_peered":"2024-05-08T21:53:23.573329+0000","last_unstale":"2024-05-08T21:53:24.566098+0000","last_undegraded":"2024-05-08T21:53:24.566098+0000","last_fullsized":"2024-05-08T21:53:24.566098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:49.915501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963958+0000","last_change":"2024-05-08T21:53:23.144147+0000","last_active":"2024-05-08T21:53:28.963958+0000","last_peered":"2024-05-08T21:53:28.963958+0000","last_clean":"2024-05-08T21:53:28.963958+0000","last_became_active":"2024-05-08T21:53:23.143808+0000","last_became_peered":"2024-05-08T21:53:23.143808+0000","last_unstale":"2024-05-08T21:53:28.963958+0000","last_undegraded":"2024-05-08T21:53:28.963958+0000","last_fullsized":"2024-05-08T21:53:28.963958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:07.011887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.592090+0000","last_change":"2024-05-08T21:53:23.583193+0000","last_active":"2024-05-08T21:53:26.592090+0000","last_peered":"2024-05-08T21:53:26.592090+0000","last_clean":"2024-05-08T21:53:26.592090+0000","last_became_active":"2024-05-08T21:53:23.582833+0000","last_became_peered":"2024-05-08T21:53:23.582833+0000","last_unstale":"2024-05-08T21:53:26.592090+0000","last_undegraded":"2024-05-08T21:53:26.592090+0000","last_fullsized":"2024-05-08T21:53:26.592090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:08.708225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":80,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:59.996776+0000","last_change":"2024-05-08T21:53:23.146398+0000","last_active":"2024-05-08T21:53:59.996776+0000","last_peered":"2024-05-08T21:53:59.996776+0000","last_clean":"2024-05-08T21:53:59.996776+0000","last_became_active":"2024-05-08T21:53:23.145823+0000","last_became_peered":"2024-05-08T21:53:23.145823+0000","last_unstale":"2024-05-08T21:53:59.996776+0000","last_undegraded":"2024-05-08T21:53:59.996776+0000","last_fullsized":"2024-05-08T21:53:59.996776+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.535292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568901+0000","last_change":"2024-05-08T21:53:23.578664+0000","last_active":"2024-05-08T21:53:24.568901+0000","last_peered":"2024-05-08T21:53:24.568901+0000","last_clean":"2024-05-08T21:53:24.568901+0000","last_became_active":"2024-05-08T21:53:23.578398+0000","last_became_peered":"2024-05-08T21:53:23.578398+0000","last_unstale":"2024-05-08T21:53:24.568901+0000","last_undegraded":"2024-05-08T21:53:24.568901+0000","last_fullsized":"2024-05-08T21:53:24.568901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:10.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.961453+0000","last_change":"2024-05-08T21:53:23.151071+0000","last_active":"2024-05-08T21:53:28.961453+0000","last_peered":"2024-05-08T21:53:28.961453+0000","last_clean":"2024-05-08T21:53:28.961453+0000","last_became_active":"2024-05-08T21:53:23.150566+0000","last_became_peered":"2024-05-08T21:53:23.150566+0000","last_unstale":"2024-05-08T21:53:28.961453+0000","last_undegraded":"2024-05-08T21:53:28.961453+0000","last_fullsized":"2024-05-08T21:53:28.961453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:51.845419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780026+0000","last_change":"2024-05-08T21:53:23.573421+0000","last_active":"2024-05-08T21:53:23.780026+0000","last_peered":"2024-05-08T21:53:23.780026+0000","last_clean":"2024-05-08T21:53:23.780026+0000","last_became_active":"2024-05-08T21:53:23.571240+0000","last_became_peered":"2024-05-08T21:53:23.571240+0000","last_unstale":"2024-05-08T21:53:23.780026+0000","last_undegraded":"2024-05-08T21:53:23.780026+0000","last_fullsized":"2024-05-08T21:53:23.780026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:51.695322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.947244+0000","last_change":"2024-05-08T21:53:23.144552+0000","last_active":"2024-05-08T21:53:27.947244+0000","last_peered":"2024-05-08T21:53:27.947244+0000","last_clean":"2024-05-08T21:53:27.947244+0000","last_became_active":"2024-05-08T21:53:23.144231+0000","last_became_peered":"2024-05-08T21:53:23.144231+0000","last_unstale":"2024-05-08T21:53:27.947244+0000","last_undegraded":"2024-05-08T21:53:27.947244+0000","last_fullsized":"2024-05-08T21:53:27.947244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:44.877035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568575+0000","last_change":"2024-05-08T21:53:23.572913+0000","last_active":"2024-05-08T21:53:24.568575+0000","last_peered":"2024-05-08T21:53:24.568575+0000","last_clean":"2024-05-08T21:53:24.568575+0000","last_became_active":"2024-05-08T21:53:23.572256+0000","last_became_peered":"2024-05-08T21:53:23.572256+0000","last_unstale":"2024-05-08T21:53:24.568575+0000","last_undegraded":"2024-05-08T21:53:24.568575+0000","last_fullsized":"2024-05-08T21:53:24.568575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:05.485571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'30","reported_seq":117,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:00.008914+0000","last_change":"2024-05-08T21:53:23.150978+0000","last_active":"2024-05-08T21:54:00.008914+0000","last_peered":"2024-05-08T21:54:00.008914+0000","last_clean":"2024-05-08T21:54:00.008914+0000","last_became_active":"2024-05-08T21:53:23.150637+0000","last_became_peered":"2024-05-08T21:53:23.150637+0000","last_unstale":"2024-05-08T21:54:00.008914+0000","last_undegraded":"2024-05-08T21:54:00.008914+0000","last_fullsized":"2024-05-08T21:54:00.008914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:40.569414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61000,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":97,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566921+0000","last_change":"2024-05-08T21:53:23.581145+0000","last_active":"2024-05-08T21:53:24.566921+0000","last_peered":"2024-05-08T21:53:24.566921+0000","last_clean":"2024-05-08T21:53:24.566921+0000","last_became_active":"2024-05-08T21:53:23.580727+0000","last_became_peered":"2024-05-08T21:53:23.580727+0000","last_unstale":"2024-05-08T21:53:24.566921+0000","last_undegraded":"2024-05-08T21:53:24.566921+0000","last_fullsized":"2024-05-08T21:53:24.566921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:58:21.737504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.962849+0000","last_change":"2024-05-08T21:53:23.146278+0000","last_active":"2024-05-08T21:53:28.962849+0000","last_peered":"2024-05-08T21:53:28.962849+0000","last_clean":"2024-05-08T21:53:28.962849+0000","last_became_active":"2024-05-08T21:53:23.145965+0000","last_became_peered":"2024-05-08T21:53:23.145965+0000","last_unstale":"2024-05-08T21:53:28.962849+0000","last_undegraded":"2024-05-08T21:53:28.962849+0000","last_fullsized":"2024-05-08T21:53:28.962849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:05.370847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806557+0000","last_change":"2024-05-08T21:53:23.576732+0000","last_active":"2024-05-08T21:53:23.806557+0000","last_peered":"2024-05-08T21:53:23.806557+0000","last_clean":"2024-05-08T21:53:23.806557+0000","last_became_active":"2024-05-08T21:53:23.576415+0000","last_became_peered":"2024-05-08T21:53:23.576415+0000","last_unstale":"2024-05-08T21:53:23.806557+0000","last_undegraded":"2024-05-08T21:53:23.806557+0000","last_fullsized":"2024-05-08T21:53:23.806557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:47.184028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963718+0000","last_change":"2024-05-08T21:53:23.148398+0000","last_active":"2024-05-08T21:53:28.963718+0000","last_peered":"2024-05-08T21:53:28.963718+0000","last_clean":"2024-05-08T21:53:28.963718+0000","last_became_active":"2024-05-08T21:53:23.148078+0000","last_became_peered":"2024-05-08T21:53:23.148078+0000","last_unstale":"2024-05-08T21:53:28.963718+0000","last_undegraded":"2024-05-08T21:53:28.963718+0000","last_fullsized":"2024-05-08T21:53:28.963718+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:38.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889064+0000","last_change":"2024-05-08T21:53:23.581700+0000","last_active":"2024-05-08T21:53:25.889064+0000","last_peered":"2024-05-08T21:53:25.889064+0000","last_clean":"2024-05-08T21:53:25.889064+0000","last_became_active":"2024-05-08T21:53:23.581346+0000","last_became_peered":"2024-05-08T21:53:23.581346+0000","last_unstale":"2024-05-08T21:53:25.889064+0000","last_undegraded":"2024-05-08T21:53:25.889064+0000","last_fullsized":"2024-05-08T21:53:25.889064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:08.530505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":57,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:01.999697+0000","last_change":"2024-05-08T21:53:23.148415+0000","last_active":"2024-05-08T21:54:01.999697+0000","last_peered":"2024-05-08T21:54:01.999697+0000","last_clean":"2024-05-08T21:54:01.999697+0000","last_became_active":"2024-05-08T21:53:23.148037+0000","last_became_peered":"2024-05-08T21:53:23.148037+0000","last_unstale":"2024-05-08T21:54:01.999697+0000","last_undegraded":"2024-05-08T21:54:01.999697+0000","last_fullsized":"2024-05-08T21:54:01.999697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:17.367009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888950+0000","last_change":"2024-05-08T21:53:23.589596+0000","last_active":"2024-05-08T21:53:25.888950+0000","last_peered":"2024-05-08T21:53:25.888950+0000","last_clean":"2024-05-08T21:53:25.888950+0000","last_became_active":"2024-05-08T21:53:23.589309+0000","last_became_peered":"2024-05-08T21:53:23.589309+0000","last_unstale":"2024-05-08T21:53:25.888950+0000","last_undegraded":"2024-05-08T21:53:25.888950+0000","last_fullsized":"2024-05-08T21:53:25.888950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:41.814532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954546+0000","last_change":"2024-05-08T21:53:23.145434+0000","last_active":"2024-05-08T21:53:27.954546+0000","last_peered":"2024-05-08T21:53:27.954546+0000","last_clean":"2024-05-08T21:53:27.954546+0000","last_became_active":"2024-05-08T21:53:23.144328+0000","last_became_peered":"2024-05-08T21:53:23.144328+0000","last_unstale":"2024-05-08T21:53:27.954546+0000","last_undegraded":"2024-05-08T21:53:27.954546+0000","last_fullsized":"2024-05-08T21:53:27.954546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:30.389806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832474+0000","last_change":"2024-05-08T21:53:23.572862+0000","last_active":"2024-05-08T21:53:24.832474+0000","last_peered":"2024-05-08T21:53:24.832474+0000","last_clean":"2024-05-08T21:53:24.832474+0000","last_became_active":"2024-05-08T21:53:23.572516+0000","last_became_peered":"2024-05-08T21:53:23.572516+0000","last_unstale":"2024-05-08T21:53:24.832474+0000","last_undegraded":"2024-05-08T21:53:24.832474+0000","last_fullsized":"2024-05-08T21:53:24.832474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:37.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.944692+0000","last_change":"2024-05-08T21:53:23.146282+0000","last_active":"2024-05-08T21:53:27.944692+0000","last_peered":"2024-05-08T21:53:27.944692+0000","last_clean":"2024-05-08T21:53:27.944692+0000","last_became_active":"2024-05-08T21:53:23.145974+0000","last_became_peered":"2024-05-08T21:53:23.145974+0000","last_unstale":"2024-05-08T21:53:27.944692+0000","last_undegraded":"2024-05-08T21:53:27.944692+0000","last_fullsized":"2024-05-08T21:53:27.944692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:26.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703149+0000","last_change":"2024-05-08T21:53:23.578164+0000","last_active":"2024-05-08T21:53:23.703149+0000","last_peered":"2024-05-08T21:53:23.703149+0000","last_clean":"2024-05-08T21:53:23.703149+0000","last_became_active":"2024-05-08T21:53:23.577832+0000","last_became_peered":"2024-05-08T21:53:23.577832+0000","last_unstale":"2024-05-08T21:53:23.703149+0000","last_undegraded":"2024-05-08T21:53:23.703149+0000","last_fullsized":"2024-05-08T21:53:23.703149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:18.700875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.592388+0000","last_change":"2024-05-08T21:53:23.151368+0000","last_active":"2024-05-08T21:53:26.592388+0000","last_peered":"2024-05-08T21:53:26.592388+0000","last_clean":"2024-05-08T21:53:26.592388+0000","last_became_active":"2024-05-08T21:53:23.151068+0000","last_became_peered":"2024-05-08T21:53:23.151068+0000","last_unstale":"2024-05-08T21:53:26.592388+0000","last_undegraded":"2024-05-08T21:53:26.592388+0000","last_fullsized":"2024-05-08T21:53:26.592388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:42.662149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.275007+0000","last_change":"2024-05-08T21:53:23.581387+0000","last_active":"2024-05-08T21:53:25.275007+0000","last_peered":"2024-05-08T21:53:25.275007+0000","last_clean":"2024-05-08T21:53:25.275007+0000","last_became_active":"2024-05-08T21:53:23.579346+0000","last_became_peered":"2024-05-08T21:53:23.579346+0000","last_unstale":"2024-05-08T21:53:25.275007+0000","last_undegraded":"2024-05-08T21:53:25.275007+0000","last_fullsized":"2024-05-08T21:53:25.275007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:42.997082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":50,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:59.018719+0000","last_change":"2024-05-08T21:53:23.150218+0000","last_active":"2024-05-08T21:53:59.018719+0000","last_peered":"2024-05-08T21:53:59.018719+0000","last_clean":"2024-05-08T21:53:59.018719+0000","last_became_active":"2024-05-08T21:53:23.148582+0000","last_became_peered":"2024-05-08T21:53:23.148582+0000","last_unstale":"2024-05-08T21:53:59.018719+0000","last_undegraded":"2024-05-08T21:53:59.018719+0000","last_fullsized":"2024-05-08T21:53:59.018719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:32.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113790+0000","last_change":"2024-05-08T21:53:23.579971+0000","last_active":"2024-05-08T21:53:25.113790+0000","last_peered":"2024-05-08T21:53:25.113790+0000","last_clean":"2024-05-08T21:53:25.113790+0000","last_became_active":"2024-05-08T21:53:23.578267+0000","last_became_peered":"2024-05-08T21:53:23.578267+0000","last_unstale":"2024-05-08T21:53:25.113790+0000","last_undegraded":"2024-05-08T21:53:25.113790+0000","last_fullsized":"2024-05-08T21:53:25.113790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:20.650914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832545+0000","last_change":"2024-05-08T21:53:23.584126+0000","last_active":"2024-05-08T21:53:24.832545+0000","last_peered":"2024-05-08T21:53:24.832545+0000","last_clean":"2024-05-08T21:53:24.832545+0000","last_became_active":"2024-05-08T21:53:23.583801+0000","last_became_peered":"2024-05-08T21:53:23.583801+0000","last_unstale":"2024-05-08T21:53:24.832545+0000","last_undegraded":"2024-05-08T21:53:24.832545+0000","last_fullsized":"2024-05-08T21:53:24.832545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:19.487948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":54,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:01.999087+0000","last_change":"2024-05-08T21:53:23.145221+0000","last_active":"2024-05-08T21:54:01.999087+0000","last_peered":"2024-05-08T21:54:01.999087+0000","last_clean":"2024-05-08T21:54:01.999087+0000","last_became_active":"2024-05-08T21:53:23.144659+0000","last_became_peered":"2024-05-08T21:53:23.144659+0000","last_unstale":"2024-05-08T21:54:01.999087+0000","last_undegraded":"2024-05-08T21:54:01.999087+0000","last_fullsized":"2024-05-08T21:54:01.999087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:58.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566506+0000","last_change":"2024-05-08T21:53:23.581748+0000","last_active":"2024-05-08T21:53:24.566506+0000","last_peered":"2024-05-08T21:53:24.566506+0000","last_clean":"2024-05-08T21:53:24.566506+0000","last_became_active":"2024-05-08T21:53:23.581336+0000","last_became_peered":"2024-05-08T21:53:23.581336+0000","last_unstale":"2024-05-08T21:53:24.566506+0000","last_undegraded":"2024-05-08T21:53:24.566506+0000","last_fullsized":"2024-05-08T21:53:24.566506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:31.158473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963098+0000","last_change":"2024-05-08T21:53:23.147073+0000","last_active":"2024-05-08T21:53:28.963098+0000","last_peered":"2024-05-08T21:53:28.963098+0000","last_clean":"2024-05-08T21:53:28.963098+0000","last_became_active":"2024-05-08T21:53:23.146720+0000","last_became_peered":"2024-05-08T21:53:23.146720+0000","last_unstale":"2024-05-08T21:53:28.963098+0000","last_undegraded":"2024-05-08T21:53:28.963098+0000","last_fullsized":"2024-05-08T21:53:28.963098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:40.566772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.977449+0000","last_change":"2024-05-08T21:52:33.304674+0000","last_active":"2024-05-08T21:53:25.977449+0000","last_peered":"2024-05-08T21:53:25.977449+0000","last_clean":"2024-05-08T21:53:25.977449+0000","last_became_active":"2024-05-08T21:52:33.304193+0000","last_became_peered":"2024-05-08T21:52:33.304193+0000","last_unstale":"2024-05-08T21:53:25.977449+0000","last_undegraded":"2024-05-08T21:53:25.977449+0000","last_fullsized":"2024-05-08T21:53:25.977449+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_clean_scrub_stamp":"2024-05-08T21:50:08.558194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:21.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806496+0000","last_change":"2024-05-08T21:53:23.583793+0000","last_active":"2024-05-08T21:53:23.806496+0000","last_peered":"2024-05-08T21:53:23.806496+0000","last_clean":"2024-05-08T21:53:23.806496+0000","last_became_active":"2024-05-08T21:53:23.583464+0000","last_became_peered":"2024-05-08T21:53:23.583464+0000","last_unstale":"2024-05-08T21:53:23.806496+0000","last_undegraded":"2024-05-08T21:53:23.806496+0000","last_fullsized":"2024-05-08T21:53:23.806496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:02.106595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.600923+0000","last_change":"2024-05-08T21:53:23.150460+0000","last_active":"2024-05-08T21:53:27.600923+0000","last_peered":"2024-05-08T21:53:27.600923+0000","last_clean":"2024-05-08T21:53:27.600923+0000","last_became_active":"2024-05-08T21:53:23.150025+0000","last_became_peered":"2024-05-08T21:53:23.150025+0000","last_unstale":"2024-05-08T21:53:27.600923+0000","last_undegraded":"2024-05-08T21:53:27.600923+0000","last_fullsized":"2024-05-08T21:53:27.600923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:28:09.311683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954963+0000","last_change":"2024-05-08T21:53:23.143160+0000","last_active":"2024-05-08T21:53:27.954963+0000","last_peered":"2024-05-08T21:53:27.954963+0000","last_clean":"2024-05-08T21:53:27.954963+0000","last_became_active":"2024-05-08T21:53:23.142718+0000","last_became_peered":"2024-05-08T21:53:23.142718+0000","last_unstale":"2024-05-08T21:53:27.954963+0000","last_undegraded":"2024-05-08T21:53:27.954963+0000","last_fullsized":"2024-05-08T21:53:27.954963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:54.148427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806605+0000","last_change":"2024-05-08T21:53:23.587069+0000","last_active":"2024-05-08T21:53:23.806605+0000","last_peered":"2024-05-08T21:53:23.806605+0000","last_clean":"2024-05-08T21:53:23.806605+0000","last_became_active":"2024-05-08T21:53:23.586782+0000","last_became_peered":"2024-05-08T21:53:23.586782+0000","last_unstale":"2024-05-08T21:53:23.806605+0000","last_undegraded":"2024-05-08T21:53:23.806605+0000","last_fullsized":"2024-05-08T21:53:23.806605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:08.897700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:32.438589+0000","last_change":"2024-05-08T21:53:23.142641+0000","last_active":"2024-05-08T21:53:32.438589+0000","last_peered":"2024-05-08T21:53:32.438589+0000","last_clean":"2024-05-08T21:53:32.438589+0000","last_became_active":"2024-05-08T21:53:23.142294+0000","last_became_peered":"2024-05-08T21:53:23.142294+0000","last_unstale":"2024-05-08T21:53:32.438589+0000","last_undegraded":"2024-05-08T21:53:32.438589+0000","last_fullsized":"2024-05-08T21:53:32.438589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:11.687406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566396+0000","last_change":"2024-05-08T21:53:23.580729+0000","last_active":"2024-05-08T21:53:24.566396+0000","last_peered":"2024-05-08T21:53:24.566396+0000","last_clean":"2024-05-08T21:53:24.566396+0000","last_became_active":"2024-05-08T21:53:23.580159+0000","last_became_peered":"2024-05-08T21:53:23.580159+0000","last_unstale":"2024-05-08T21:53:24.566396+0000","last_undegraded":"2024-05-08T21:53:24.566396+0000","last_fullsized":"2024-05-08T21:53:24.566396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:43:33.336308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963066+0000","last_change":"2024-05-08T21:53:23.144242+0000","last_active":"2024-05-08T21:53:28.963066+0000","last_peered":"2024-05-08T21:53:28.963066+0000","last_clean":"2024-05-08T21:53:28.963066+0000","last_became_active":"2024-05-08T21:53:23.143905+0000","last_became_peered":"2024-05-08T21:53:23.143905+0000","last_unstale":"2024-05-08T21:53:28.963066+0000","last_undegraded":"2024-05-08T21:53:28.963066+0000","last_fullsized":"2024-05-08T21:53:28.963066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:41.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779991+0000","last_change":"2024-05-08T21:53:23.576972+0000","last_active":"2024-05-08T21:53:23.779991+0000","last_peered":"2024-05-08T21:53:23.779991+0000","last_clean":"2024-05-08T21:53:23.779991+0000","last_became_active":"2024-05-08T21:53:23.576668+0000","last_became_peered":"2024-05-08T21:53:23.576668+0000","last_unstale":"2024-05-08T21:53:23.779991+0000","last_undegraded":"2024-05-08T21:53:23.779991+0000","last_fullsized":"2024-05-08T21:53:23.779991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:25.545108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.595186+0000","last_change":"2024-05-08T21:53:23.145610+0000","last_active":"2024-05-08T21:53:26.595186+0000","last_peered":"2024-05-08T21:53:26.595186+0000","last_clean":"2024-05-08T21:53:26.595186+0000","last_became_active":"2024-05-08T21:53:23.145075+0000","last_became_peered":"2024-05-08T21:53:23.145075+0000","last_unstale":"2024-05-08T21:53:26.595186+0000","last_undegraded":"2024-05-08T21:53:26.595186+0000","last_fullsized":"2024-05-08T21:53:26.595186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:58.351303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566650+0000","last_change":"2024-05-08T21:53:23.579335+0000","last_active":"2024-05-08T21:53:24.566650+0000","last_peered":"2024-05-08T21:53:24.566650+0000","last_clean":"2024-05-08T21:53:24.566650+0000","last_became_active":"2024-05-08T21:53:23.578956+0000","last_became_peered":"2024-05-08T21:53:23.578956+0000","last_unstale":"2024-05-08T21:53:24.566650+0000","last_undegraded":"2024-05-08T21:53:24.566650+0000","last_fullsized":"2024-05-08T21:53:24.566650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:34.872954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.964020+0000","last_change":"2024-05-08T21:53:23.144108+0000","last_active":"2024-05-08T21:53:28.964020+0000","last_peered":"2024-05-08T21:53:28.964020+0000","last_clean":"2024-05-08T21:53:28.964020+0000","last_became_active":"2024-05-08T21:53:23.143786+0000","last_became_peered":"2024-05-08T21:53:23.143786+0000","last_unstale":"2024-05-08T21:53:28.964020+0000","last_undegraded":"2024-05-08T21:53:28.964020+0000","last_fullsized":"2024-05-08T21:53:28.964020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:15.512879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703096+0000","last_change":"2024-05-08T21:53:23.587667+0000","last_active":"2024-05-08T21:53:23.703096+0000","last_peered":"2024-05-08T21:53:23.703096+0000","last_clean":"2024-05-08T21:53:23.703096+0000","last_became_active":"2024-05-08T21:53:23.586908+0000","last_became_peered":"2024-05-08T21:53:23.586908+0000","last_unstale":"2024-05-08T21:53:23.703096+0000","last_undegraded":"2024-05-08T21:53:23.703096+0000","last_fullsized":"2024-05-08T21:53:23.703096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:06.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:32.081481+0000","last_change":"2024-05-08T21:53:23.150602+0000","last_active":"2024-05-08T21:53:32.081481+0000","last_peered":"2024-05-08T21:53:32.081481+0000","last_clean":"2024-05-08T21:53:32.081481+0000","last_became_active":"2024-05-08T21:53:23.150256+0000","last_became_peered":"2024-05-08T21:53:23.150256+0000","last_unstale":"2024-05-08T21:53:32.081481+0000","last_undegraded":"2024-05-08T21:53:32.081481+0000","last_fullsized":"2024-05-08T21:53:32.081481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:22:05.088246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566829+0000","last_change":"2024-05-08T21:53:23.577768+0000","last_active":"2024-05-08T21:53:24.566829+0000","last_peered":"2024-05-08T21:53:24.566829+0000","last_clean":"2024-05-08T21:53:24.566829+0000","last_became_active":"2024-05-08T21:53:23.577388+0000","last_became_peered":"2024-05-08T21:53:23.577388+0000","last_unstale":"2024-05-08T21:53:24.566829+0000","last_undegraded":"2024-05-08T21:53:24.566829+0000","last_fullsized":"2024-05-08T21:53:24.566829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:24.707904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.593957+0000","last_change":"2024-05-08T21:53:23.145165+0000","last_active":"2024-05-08T21:53:26.593957+0000","last_peered":"2024-05-08T21:53:26.593957+0000","last_clean":"2024-05-08T21:53:26.593957+0000","last_became_active":"2024-05-08T21:53:23.144465+0000","last_became_peered":"2024-05-08T21:53:23.144465+0000","last_unstale":"2024-05-08T21:53:26.593957+0000","last_undegraded":"2024-05-08T21:53:26.593957+0000","last_fullsized":"2024-05-08T21:53:26.593957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:47.229905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889021+0000","last_change":"2024-05-08T21:53:23.583476+0000","last_active":"2024-05-08T21:53:25.889021+0000","last_peered":"2024-05-08T21:53:25.889021+0000","last_clean":"2024-05-08T21:53:25.889021+0000","last_became_active":"2024-05-08T21:53:23.582989+0000","last_became_peered":"2024-05-08T21:53:23.582989+0000","last_unstale":"2024-05-08T21:53:25.889021+0000","last_undegraded":"2024-05-08T21:53:25.889021+0000","last_fullsized":"2024-05-08T21:53:25.889021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:05.402755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":53,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:02.034414+0000","last_change":"2024-05-08T21:53:23.147469+0000","last_active":"2024-05-08T21:54:02.034414+0000","last_peered":"2024-05-08T21:54:02.034414+0000","last_clean":"2024-05-08T21:54:02.034414+0000","last_became_active":"2024-05-08T21:53:23.147145+0000","last_became_peered":"2024-05-08T21:53:23.147145+0000","last_unstale":"2024-05-08T21:54:02.034414+0000","last_undegraded":"2024-05-08T21:54:02.034414+0000","last_fullsized":"2024-05-08T21:54:02.034414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:15.806118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889101+0000","last_change":"2024-05-08T21:53:23.583221+0000","last_active":"2024-05-08T21:53:25.889101+0000","last_peered":"2024-05-08T21:53:25.889101+0000","last_clean":"2024-05-08T21:53:25.889101+0000","last_became_active":"2024-05-08T21:53:23.582942+0000","last_became_peered":"2024-05-08T21:53:23.582942+0000","last_unstale":"2024-05-08T21:53:25.889101+0000","last_undegraded":"2024-05-08T21:53:25.889101+0000","last_fullsized":"2024-05-08T21:53:25.889101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:32.188611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953180+0000","last_change":"2024-05-08T21:53:23.143721+0000","last_active":"2024-05-08T21:53:27.953180+0000","last_peered":"2024-05-08T21:53:27.953180+0000","last_clean":"2024-05-08T21:53:27.953180+0000","last_became_active":"2024-05-08T21:53:23.143227+0000","last_became_peered":"2024-05-08T21:53:23.143227+0000","last_unstale":"2024-05-08T21:53:27.953180+0000","last_undegraded":"2024-05-08T21:53:27.953180+0000","last_fullsized":"2024-05-08T21:53:27.953180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:09.619865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.977682+0000","last_change":"2024-05-08T21:53:23.585820+0000","last_active":"2024-05-08T21:53:25.977682+0000","last_peered":"2024-05-08T21:53:25.977682+0000","last_clean":"2024-05-08T21:53:25.977682+0000","last_became_active":"2024-05-08T21:53:23.585435+0000","last_became_peered":"2024-05-08T21:53:23.585435+0000","last_unstale":"2024-05-08T21:53:25.977682+0000","last_undegraded":"2024-05-08T21:53:25.977682+0000","last_fullsized":"2024-05-08T21:53:25.977682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:48.696651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963041+0000","last_change":"2024-05-08T21:53:23.141992+0000","last_active":"2024-05-08T21:53:28.963041+0000","last_peered":"2024-05-08T21:53:28.963041+0000","last_clean":"2024-05-08T21:53:28.963041+0000","last_became_active":"2024-05-08T21:53:23.141670+0000","last_became_peered":"2024-05-08T21:53:23.141670+0000","last_unstale":"2024-05-08T21:53:28.963041+0000","last_undegraded":"2024-05-08T21:53:28.963041+0000","last_fullsized":"2024-05-08T21:53:28.963041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:53.191681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889139+0000","last_change":"2024-05-08T21:53:23.586649+0000","last_active":"2024-05-08T21:53:25.889139+0000","last_peered":"2024-05-08T21:53:25.889139+0000","last_clean":"2024-05-08T21:53:25.889139+0000","last_became_active":"2024-05-08T21:53:23.586069+0000","last_became_peered":"2024-05-08T21:53:23.586069+0000","last_unstale":"2024-05-08T21:53:25.889139+0000","last_undegraded":"2024-05-08T21:53:25.889139+0000","last_fullsized":"2024-05-08T21:53:25.889139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:59.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703112+0000","last_change":"2024-05-08T21:53:23.588268+0000","last_active":"2024-05-08T21:53:23.703112+0000","last_peered":"2024-05-08T21:53:23.703112+0000","last_clean":"2024-05-08T21:53:23.703112+0000","last_became_active":"2024-05-08T21:53:23.587858+0000","last_became_peered":"2024-05-08T21:53:23.587858+0000","last_unstale":"2024-05-08T21:53:23.703112+0000","last_undegraded":"2024-05-08T21:53:23.703112+0000","last_fullsized":"2024-05-08T21:53:23.703112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:47.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.589133+0000","last_change":"2024-05-08T21:53:23.587227+0000","last_active":"2024-05-08T21:53:23.589133+0000","last_peered":"2024-05-08T21:53:23.589133+0000","last_clean":"2024-05-08T21:53:23.589133+0000","last_became_active":"2024-05-08T21:53:23.586807+0000","last_became_peered":"2024-05-08T21:53:23.586807+0000","last_unstale":"2024-05-08T21:53:23.589133+0000","last_undegraded":"2024-05-08T21:53:23.589133+0000","last_fullsized":"2024-05-08T21:53:23.589133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:29.814057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585531+0000","last_change":"2024-05-08T21:53:23.583968+0000","last_active":"2024-05-08T21:53:23.585531+0000","last_peered":"2024-05-08T21:53:23.585531+0000","last_clean":"2024-05-08T21:53:23.585531+0000","last_became_active":"2024-05-08T21:53:23.583625+0000","last_became_peered":"2024-05-08T21:53:23.583625+0000","last_unstale":"2024-05-08T21:53:23.585531+0000","last_undegraded":"2024-05-08T21:53:23.585531+0000","last_fullsized":"2024-05-08T21:53:23.585531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:40.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779949+0000","last_change":"2024-05-08T21:53:23.571284+0000","last_active":"2024-05-08T21:53:23.779949+0000","last_peered":"2024-05-08T21:53:23.779949+0000","last_clean":"2024-05-08T21:53:23.779949+0000","last_became_active":"2024-05-08T21:53:23.570915+0000","last_became_peered":"2024-05-08T21:53:23.570915+0000","last_unstale":"2024-05-08T21:53:23.779949+0000","last_undegraded":"2024-05-08T21:53:23.779949+0000","last_fullsized":"2024-05-08T21:53:23.779949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:52.562178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806616+0000","last_change":"2024-05-08T21:53:23.572011+0000","last_active":"2024-05-08T21:53:23.806616+0000","last_peered":"2024-05-08T21:53:23.806616+0000","last_clean":"2024-05-08T21:53:23.806616+0000","last_became_active":"2024-05-08T21:53:23.571788+0000","last_became_peered":"2024-05-08T21:53:23.571788+0000","last_unstale":"2024-05-08T21:53:23.806616+0000","last_undegraded":"2024-05-08T21:53:23.806616+0000","last_fullsized":"2024-05-08T21:53:23.806616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:43.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703137+0000","last_change":"2024-05-08T21:53:23.582823+0000","last_active":"2024-05-08T21:53:23.703137+0000","last_peered":"2024-05-08T21:53:23.703137+0000","last_clean":"2024-05-08T21:53:23.703137+0000","last_became_active":"2024-05-08T21:53:23.582366+0000","last_became_peered":"2024-05-08T21:53:23.582366+0000","last_unstale":"2024-05-08T21:53:23.703137+0000","last_undegraded":"2024-05-08T21:53:23.703137+0000","last_fullsized":"2024-05-08T21:53:23.703137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:44.029915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585670+0000","last_change":"2024-05-08T21:53:23.583566+0000","last_active":"2024-05-08T21:53:23.585670+0000","last_peered":"2024-05-08T21:53:23.585670+0000","last_clean":"2024-05-08T21:53:23.585670+0000","last_became_active":"2024-05-08T21:53:23.583204+0000","last_became_peered":"2024-05-08T21:53:23.583204+0000","last_unstale":"2024-05-08T21:53:23.585670+0000","last_undegraded":"2024-05-08T21:53:23.585670+0000","last_fullsized":"2024-05-08T21:53:23.585670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:41.426005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566280+0000","last_change":"2024-05-08T21:53:23.587105+0000","last_active":"2024-05-08T21:53:24.566280+0000","last_peered":"2024-05-08T21:53:24.566280+0000","last_clean":"2024-05-08T21:53:24.566280+0000","last_became_active":"2024-05-08T21:53:23.586773+0000","last_became_peered":"2024-05-08T21:53:23.586773+0000","last_unstale":"2024-05-08T21:53:24.566280+0000","last_undegraded":"2024-05-08T21:53:24.566280+0000","last_fullsized":"2024-05-08T21:53:24.566280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:50.809186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832696+0000","last_change":"2024-05-08T21:53:23.576779+0000","last_active":"2024-05-08T21:53:24.832696+0000","last_peered":"2024-05-08T21:53:24.832696+0000","last_clean":"2024-05-08T21:53:24.832696+0000","last_became_active":"2024-05-08T21:53:23.576434+0000","last_became_peered":"2024-05-08T21:53:23.576434+0000","last_unstale":"2024-05-08T21:53:24.832696+0000","last_undegraded":"2024-05-08T21:53:24.832696+0000","last_fullsized":"2024-05-08T21:53:24.832696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:10.168522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566243+0000","last_change":"2024-05-08T21:53:23.585196+0000","last_active":"2024-05-08T21:53:24.566243+0000","last_peered":"2024-05-08T21:53:24.566243+0000","last_clean":"2024-05-08T21:53:24.566243+0000","last_became_active":"2024-05-08T21:53:23.584160+0000","last_became_peered":"2024-05-08T21:53:23.584160+0000","last_unstale":"2024-05-08T21:53:24.566243+0000","last_undegraded":"2024-05-08T21:53:24.566243+0000","last_fullsized":"2024-05-08T21:53:24.566243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:15.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568545+0000","last_change":"2024-05-08T21:53:23.577887+0000","last_active":"2024-05-08T21:53:24.568545+0000","last_peered":"2024-05-08T21:53:24.568545+0000","last_clean":"2024-05-08T21:53:24.568545+0000","last_became_active":"2024-05-08T21:53:23.577597+0000","last_became_peered":"2024-05-08T21:53:23.577597+0000","last_unstale":"2024-05-08T21:53:24.568545+0000","last_undegraded":"2024-05-08T21:53:24.568545+0000","last_fullsized":"2024-05-08T21:53:24.568545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:33.704002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806594+0000","last_change":"2024-05-08T21:53:23.575180+0000","last_active":"2024-05-08T21:53:23.806594+0000","last_peered":"2024-05-08T21:53:23.806594+0000","last_clean":"2024-05-08T21:53:23.806594+0000","last_became_active":"2024-05-08T21:53:23.574521+0000","last_became_peered":"2024-05-08T21:53:23.574521+0000","last_unstale":"2024-05-08T21:53:23.806594+0000","last_undegraded":"2024-05-08T21:53:23.806594+0000","last_fullsized":"2024-05-08T21:53:23.806594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:59.609492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703125+0000","last_change":"2024-05-08T21:53:23.585380+0000","last_active":"2024-05-08T21:53:23.703125+0000","last_peered":"2024-05-08T21:53:23.703125+0000","last_clean":"2024-05-08T21:53:23.703125+0000","last_became_active":"2024-05-08T21:53:23.585073+0000","last_became_peered":"2024-05-08T21:53:23.585073+0000","last_unstale":"2024-05-08T21:53:23.703125+0000","last_undegraded":"2024-05-08T21:53:23.703125+0000","last_fullsized":"2024-05-08T21:53:23.703125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:33.572410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68409,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":373,"num_read_kb":262,"num_write":232,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":860160,"data_stored":410274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.495}]},{"osd":1,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69899999999999995}]},{"osd":2,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.71499999999999997}]},{"osd":7,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.59899999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.624},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.74299999999999999}]}]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32632,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712520,"statfs":{"total":95995035648,"available":95961620480,"internally_reserved":0,"allocated":1556480,"data_stored":1062365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.745},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.72899999999999998}]},{"osd":3,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1,"5min":1,"15min":1},"last":0.80900000000000005}]},{"osd":4,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85799999999999998}]},{"osd":6,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.877},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85199999999999998}]},{"osd":9,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82799999999999996}]}]},{"osd":9,"up_from":62,"seq":266287972377,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":410252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34300000000000003}]},{"osd":1,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.502},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439}]},{"osd":3,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.38}]},{"osd":4,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.54200000000000004}]},{"osd":6,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57399999999999995}]},{"osd":8,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56499999999999995}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.60099999999999998}]}]},{"osd":8,"up_from":55,"seq":236223201309,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":410299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.59399999999999997,"15min":0.56799999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.0030000000000001,"5min":5.0030000000000001,"15min":5.0030000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60099999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.432,"15min":0.41699999999999998},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.496},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47499999999999998,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92100000000000004,"5min":0.996,"15min":0.996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83399999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.68200000000000005,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8819999999999997,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.69899999999999995}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed May 8 21:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.748},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.875}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430239,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1478656,"data_stored":1003792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.442,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44400000000000001}]},{"osd":2,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84499999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67200000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82699999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66900000000000004}]},{"osd":6,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.49199999999999999}]},{"osd":8,"last update":"Wed May 8 21:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93700000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65300000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68400000000000005}]},{"osd":10,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.621}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":6,"up_from":43,"seq":184683593764,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":410314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.28899999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.31}]},{"osd":1,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.437},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.871,"15min":0.871},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed May 8 21:53:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 21:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80200000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.873,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.45800000000000002}]},{"osd":10,"last update":"Wed May 8 21:53:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":472227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.439},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.78500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed May 8 21:53:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.749,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed May 8 21:53:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:44 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.41699999999999998,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.309,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.33700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.83599999999999997}]},{"osd":6,"last update":"Wed May 8 21:53:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.752}]},{"osd":7,"last update":"Wed May 8 21:53:38 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.50800000000000001,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.255,"15min":0.255},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.503,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 21:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.433,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.65000000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed May 8 21:53:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":411260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.53800000000000003,"15min":0.48199999999999998},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.50600000000000001,"15min":0.47199999999999998},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.5}]},{"osd":2,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.83599999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed May 8 21:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.47799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.625}]},{"osd":6,"last update":"Wed May 8 21:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41999999999999998}]},{"osd":7,"last update":"Wed May 8 21:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73199999999999998}]},{"osd":8,"last update":"Wed May 8 21:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58599999999999997}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":10,"last update":"Wed May 8 21:53:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67300000000000004}]},{"osd":11,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66200000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":472399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48999999999999999,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69799999999999995}]},{"osd":3,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73899999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed May 8 21:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed May 8 21:53:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71999999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82799999999999996,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52900000000000003}]},{"osd":8,"last update":"Wed May 8 21:54:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.62},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60599999999999998}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":3,"up_from":24,"seq":103079215151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":884736,"data_stored":409794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.34999999999999998},"min":{"1min":0.28000000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65100000000000002,"5min":1.028,"15min":1.028},"last":0.27700000000000002}]},{"osd":1,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.433,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51100000000000001}]},{"osd":4,"last update":"Wed May 8 21:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.873,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.873,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed May 8 21:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42199999999999999}]},{"osd":7,"last update":"Wed May 8 21:53:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80800000000000005,"5min":1.056,"15min":1.056},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.54100000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92100000000000004,"5min":1.329,"15min":1.329},"last":0.47999999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.627},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.48599999999999999}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.438},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.62}]},{"osd":10,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":410175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed May 8 21:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.28000000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.68799999999999994,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.623,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed May 8 21:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71299999999999997}]},{"osd":3,"last update":"Wed May 8 21:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.747},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed May 8 21:53:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39900000000000002}]},{"osd":8,"last update":"Wed May 8 21:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32564,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712588,"statfs":{"total":95995035648,"available":95961690112,"internally_reserved":0,"allocated":1486848,"data_stored":1002449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.504},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.498},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.37,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.253,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.436},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48699999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.47299999999999998}]},{"osd":3,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.78100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed May 8 21:53:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59899999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed May 8 21:53:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed May 8 21:53:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.57199999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64800000000000002}]},{"osd":8,"last update":"Wed May 8 21:53:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.41399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.74199999999999999}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.621},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68100000000000005}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.6499999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.617,"5min":3.617,"15min":3.617},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:54:05.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T21:54:06.144 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:06.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:06 smithi064 ceph-mon[42500]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:54:06.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:06 smithi160 ceph-mon[46962]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:54:06.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:06 smithi186 ceph-mon[46619]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 -- 172.21.15.64:0/3964258113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8104ed0 msgr2=0x7fced81052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 --2- 172.21.15.64:0/3964258113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8104ed0 0x7fced81052b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0099c0 tx=0x7fcecc009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 -- 172.21.15.64:0/3964258113 shutdown_connections 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 --2- 172.21.15.64:0/3964258113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced8101410 0x7fced810cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 --2- 172.21.15.64:0/3964258113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8104ed0 0x7fced81052b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 --2- 172.21.15.64:0/3964258113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 0x7fced8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 -- 172.21.15.64:0/3964258113 >> 172.21.15.64:0/3964258113 conn(0x7fced8075260 msgr2=0x7fced8075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 -- 172.21.15.64:0/3964258113 shutdown_connections 2024-05-08T21:54:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.201+0000 7fcedf183700 1 -- 172.21.15.64:0/3964258113 wait complete. 2024-05-08T21:54:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 Processor -- start 2024-05-08T21:54:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 -- start start 2024-05-08T21:54:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 0x7fced8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 0x7fced8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced8104ed0 0x7fced819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced819e3d0 con 0x7fced8101410 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced819e540 con 0x7fced8100a50 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedf183700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced819e700 con 0x7fced8104ed0 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 0x7fced8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 0x7fced8199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50448/0 (socket says 172.21.15.64:50448) 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 -- 172.21.15.64:0/3110420011 learned_addr learned my addr 172.21.15.64:0/3110420011 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedcf1f700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 0x7fced8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedd720700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced8104ed0 0x7fced819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 -- 172.21.15.64:0/3110420011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced8104ed0 msgr2=0x7fced819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced8104ed0 0x7fced819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 -- 172.21.15.64:0/3110420011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 msgr2=0x7fced8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 0x7fced8199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fced7fff700 1 -- 172.21.15.64:0/3110420011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0095f0 con 0x7fced8101410 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.202+0000 7fcedcf1f700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 0x7fced8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:07.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.203+0000 7fced7fff700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 0x7fced8199a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcec400d950 tx=0x7fcec400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.203+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4009b40 con 0x7fced8101410 2024-05-08T21:54:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.203+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced819e9e0 con 0x7fced8101410 2024-05-08T21:54:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.203+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcec4009ca0 con 0x7fced8101410 2024-05-08T21:54:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.203+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec400f620 con 0x7fced8101410 2024-05-08T21:54:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.203+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced81a6240 con 0x7fced8101410 2024-05-08T21:54:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.204+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced80fef50 con 0x7fced8101410 2024-05-08T21:54:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.205+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcec4010460 con 0x7fced8101410 2024-05-08T21:54:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.206+0000 7fced5ffb700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcec80a11b0 0x7fcec80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.206+0000 7fcedcf1f700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcec80a11b0 0x7fcec80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.206+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcec40ccc60 con 0x7fced8101410 2024-05-08T21:54:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.206+0000 7fcedcf1f700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcec80a11b0 0x7fcec80a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009fd0 tx=0x7fcecc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.213+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcec4096440 con 0x7fced8101410 2024-05-08T21:54:07.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.412+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fced819a7e0 con 0x7fcec80a11b0 2024-05-08T21:54:07.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.422+0000 7fced5ffb700 1 -- 172.21.15.64:0/3110420011 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340578 (secure 0 0 0) 0x7fced819a7e0 con 0x7fcec80a11b0 2024-05-08T21:54:07.428 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:07.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcec80a11b0 msgr2=0x7fcec80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:07.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcec80a11b0 0x7fcec80a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009fd0 tx=0x7fcecc005fd0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 msgr2=0x7fced8199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:07.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 0x7fced8199a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcec400d950 tx=0x7fcec400dd10 comp rx=0 tx=0).stop 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 shutdown_connections 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcec80a11b0 0x7fcec80a3670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8100a50 0x7fced8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced8104ed0 0x7fced819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 --2- 172.21.15.64:0/3110420011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced8101410 0x7fced8199a60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 >> 172.21.15.64:0/3110420011 conn(0x7fced8075260 msgr2=0x7fced810b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 shutdown_connections 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:07.426+0000 7fcedf183700 1 -- 172.21.15.64:0/3110420011 wait complete. 2024-05-08T21:54:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-08T21:54:07.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:07 smithi064 ceph-mon[42500]: pgmap v242: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:07.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:07 smithi160 ceph-mon[46962]: pgmap v242: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:07 smithi186 ceph-mon[46619]: pgmap v242: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:07.811 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-05-08T21:54:06.103861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659011,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":529,"num_read_kb":386,"num_write":415,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356912,"kb_used_data":12240,"kb_used_omap":18,"kb_used_meta":344557,"kb_avail":1124584912,"statfs":{"total":1151940427776,"available":1151574949888,"internally_reserved":0,"allocated":12533760,"data_stored":6885600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":352826693},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":67,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703161+0000","last_change":"2024-05-08T21:53:23.579256+0000","last_active":"2024-05-08T21:53:23.703161+0000","last_peered":"2024-05-08T21:53:23.703161+0000","last_clean":"2024-05-08T21:53:23.703161+0000","last_became_active":"2024-05-08T21:53:23.578867+0000","last_became_peered":"2024-05-08T21:53:23.578867+0000","last_unstale":"2024-05-08T21:53:23.703161+0000","last_undegraded":"2024-05-08T21:53:23.703161+0000","last_fullsized":"2024-05-08T21:53:23.703161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:07.123241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780009+0000","last_change":"2024-05-08T21:53:23.575229+0000","last_active":"2024-05-08T21:53:23.780009+0000","last_peered":"2024-05-08T21:53:23.780009+0000","last_clean":"2024-05-08T21:53:23.780009+0000","last_became_active":"2024-05-08T21:53:23.574933+0000","last_became_peered":"2024-05-08T21:53:23.574933+0000","last_unstale":"2024-05-08T21:53:23.780009+0000","last_undegraded":"2024-05-08T21:53:23.780009+0000","last_fullsized":"2024-05-08T21:53:23.780009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:04.592572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779970+0000","last_change":"2024-05-08T21:53:23.587183+0000","last_active":"2024-05-08T21:53:23.779970+0000","last_peered":"2024-05-08T21:53:23.779970+0000","last_clean":"2024-05-08T21:53:23.779970+0000","last_became_active":"2024-05-08T21:53:23.586861+0000","last_became_peered":"2024-05-08T21:53:23.586861+0000","last_unstale":"2024-05-08T21:53:23.779970+0000","last_undegraded":"2024-05-08T21:53:23.779970+0000","last_fullsized":"2024-05-08T21:53:23.779970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:48.878049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.587773+0000","last_change":"2024-05-08T21:53:23.587097+0000","last_active":"2024-05-08T21:53:23.587773+0000","last_peered":"2024-05-08T21:53:23.587773+0000","last_clean":"2024-05-08T21:53:23.587773+0000","last_became_active":"2024-05-08T21:53:23.586756+0000","last_became_peered":"2024-05-08T21:53:23.586756+0000","last_unstale":"2024-05-08T21:53:23.587773+0000","last_undegraded":"2024-05-08T21:53:23.587773+0000","last_fullsized":"2024-05-08T21:53:23.587773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:38.938727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566720+0000","last_change":"2024-05-08T21:53:23.581157+0000","last_active":"2024-05-08T21:53:24.566720+0000","last_peered":"2024-05-08T21:53:24.566720+0000","last_clean":"2024-05-08T21:53:24.566720+0000","last_became_active":"2024-05-08T21:53:23.580803+0000","last_became_peered":"2024-05-08T21:53:23.580803+0000","last_unstale":"2024-05-08T21:53:24.566720+0000","last_undegraded":"2024-05-08T21:53:24.566720+0000","last_fullsized":"2024-05-08T21:53:24.566720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:13:59.878096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113677+0000","last_change":"2024-05-08T21:53:23.583093+0000","last_active":"2024-05-08T21:53:25.113677+0000","last_peered":"2024-05-08T21:53:25.113677+0000","last_clean":"2024-05-08T21:53:25.113677+0000","last_became_active":"2024-05-08T21:53:23.582798+0000","last_became_peered":"2024-05-08T21:53:23.582798+0000","last_unstale":"2024-05-08T21:53:25.113677+0000","last_undegraded":"2024-05-08T21:53:25.113677+0000","last_fullsized":"2024-05-08T21:53:25.113677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:53.682817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888997+0000","last_change":"2024-05-08T21:53:23.586322+0000","last_active":"2024-05-08T21:53:25.888997+0000","last_peered":"2024-05-08T21:53:25.888997+0000","last_clean":"2024-05-08T21:53:25.888997+0000","last_became_active":"2024-05-08T21:53:23.585938+0000","last_became_peered":"2024-05-08T21:53:23.585938+0000","last_unstale":"2024-05-08T21:53:25.888997+0000","last_undegraded":"2024-05-08T21:53:25.888997+0000","last_fullsized":"2024-05-08T21:53:25.888997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:22.744048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566197+0000","last_change":"2024-05-08T21:53:23.580496+0000","last_active":"2024-05-08T21:53:24.566197+0000","last_peered":"2024-05-08T21:53:24.566197+0000","last_clean":"2024-05-08T21:53:24.566197+0000","last_became_active":"2024-05-08T21:53:23.580154+0000","last_became_peered":"2024-05-08T21:53:23.580154+0000","last_unstale":"2024-05-08T21:53:24.566197+0000","last_undegraded":"2024-05-08T21:53:24.566197+0000","last_fullsized":"2024-05-08T21:53:24.566197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:50.713758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568936+0000","last_change":"2024-05-08T21:53:23.580549+0000","last_active":"2024-05-08T21:53:24.568936+0000","last_peered":"2024-05-08T21:53:24.568936+0000","last_clean":"2024-05-08T21:53:24.568936+0000","last_became_active":"2024-05-08T21:53:23.579962+0000","last_became_peered":"2024-05-08T21:53:23.579962+0000","last_unstale":"2024-05-08T21:53:24.568936+0000","last_undegraded":"2024-05-08T21:53:24.568936+0000","last_fullsized":"2024-05-08T21:53:24.568936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:52.010295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113920+0000","last_change":"2024-05-08T21:53:23.589640+0000","last_active":"2024-05-08T21:53:25.113920+0000","last_peered":"2024-05-08T21:53:25.113920+0000","last_clean":"2024-05-08T21:53:25.113920+0000","last_became_active":"2024-05-08T21:53:23.589347+0000","last_became_peered":"2024-05-08T21:53:23.589347+0000","last_unstale":"2024-05-08T21:53:25.113920+0000","last_undegraded":"2024-05-08T21:53:25.113920+0000","last_fullsized":"2024-05-08T21:53:25.113920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:40.346414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889087+0000","last_change":"2024-05-08T21:53:23.586430+0000","last_active":"2024-05-08T21:53:25.889087+0000","last_peered":"2024-05-08T21:53:25.889087+0000","last_clean":"2024-05-08T21:53:25.889087+0000","last_became_active":"2024-05-08T21:53:23.585986+0000","last_became_peered":"2024-05-08T21:53:23.585986+0000","last_unstale":"2024-05-08T21:53:25.889087+0000","last_undegraded":"2024-05-08T21:53:25.889087+0000","last_fullsized":"2024-05-08T21:53:25.889087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:09.970481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.586950+0000","last_change":"2024-05-08T21:53:23.585441+0000","last_active":"2024-05-08T21:53:23.586950+0000","last_peered":"2024-05-08T21:53:23.586950+0000","last_clean":"2024-05-08T21:53:23.586950+0000","last_became_active":"2024-05-08T21:53:23.584809+0000","last_became_peered":"2024-05-08T21:53:23.584809+0000","last_unstale":"2024-05-08T21:53:23.586950+0000","last_undegraded":"2024-05-08T21:53:23.586950+0000","last_fullsized":"2024-05-08T21:53:23.586950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:31.599185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806575+0000","last_change":"2024-05-08T21:53:23.575757+0000","last_active":"2024-05-08T21:53:23.806575+0000","last_peered":"2024-05-08T21:53:23.806575+0000","last_clean":"2024-05-08T21:53:23.806575+0000","last_became_active":"2024-05-08T21:53:23.575376+0000","last_became_peered":"2024-05-08T21:53:23.575376+0000","last_unstale":"2024-05-08T21:53:23.806575+0000","last_undegraded":"2024-05-08T21:53:23.806575+0000","last_fullsized":"2024-05-08T21:53:23.806575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.851616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779926+0000","last_change":"2024-05-08T21:53:23.573225+0000","last_active":"2024-05-08T21:53:23.779926+0000","last_peered":"2024-05-08T21:53:23.779926+0000","last_clean":"2024-05-08T21:53:23.779926+0000","last_became_active":"2024-05-08T21:53:23.572784+0000","last_became_peered":"2024-05-08T21:53:23.572784+0000","last_unstale":"2024-05-08T21:53:23.779926+0000","last_undegraded":"2024-05-08T21:53:23.779926+0000","last_fullsized":"2024-05-08T21:53:23.779926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:11.963959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779898+0000","last_change":"2024-05-08T21:53:23.583961+0000","last_active":"2024-05-08T21:53:23.779898+0000","last_peered":"2024-05-08T21:53:23.779898+0000","last_clean":"2024-05-08T21:53:23.779898+0000","last_became_active":"2024-05-08T21:53:23.583532+0000","last_became_peered":"2024-05-08T21:53:23.583532+0000","last_unstale":"2024-05-08T21:53:23.779898+0000","last_undegraded":"2024-05-08T21:53:23.779898+0000","last_fullsized":"2024-05-08T21:53:23.779898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:15:04.318298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568388+0000","last_change":"2024-05-08T21:53:23.577361+0000","last_active":"2024-05-08T21:53:24.568388+0000","last_peered":"2024-05-08T21:53:24.568388+0000","last_clean":"2024-05-08T21:53:24.568388+0000","last_became_active":"2024-05-08T21:53:23.577003+0000","last_became_peered":"2024-05-08T21:53:23.577003+0000","last_unstale":"2024-05-08T21:53:24.568388+0000","last_undegraded":"2024-05-08T21:53:24.568388+0000","last_fullsized":"2024-05-08T21:53:24.568388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:14.617211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703171+0000","last_change":"2024-05-08T21:53:23.579881+0000","last_active":"2024-05-08T21:53:23.703171+0000","last_peered":"2024-05-08T21:53:23.703171+0000","last_clean":"2024-05-08T21:53:23.703171+0000","last_became_active":"2024-05-08T21:53:23.579569+0000","last_became_peered":"2024-05-08T21:53:23.579569+0000","last_unstale":"2024-05-08T21:53:23.703171+0000","last_undegraded":"2024-05-08T21:53:23.703171+0000","last_fullsized":"2024-05-08T21:53:23.703171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:38.210018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.591923+0000","last_change":"2024-05-08T21:53:23.573027+0000","last_active":"2024-05-08T21:53:26.591923+0000","last_peered":"2024-05-08T21:53:26.591923+0000","last_clean":"2024-05-08T21:53:26.591923+0000","last_became_active":"2024-05-08T21:53:23.572709+0000","last_became_peered":"2024-05-08T21:53:23.572709+0000","last_unstale":"2024-05-08T21:53:26.591923+0000","last_undegraded":"2024-05-08T21:53:26.591923+0000","last_fullsized":"2024-05-08T21:53:26.591923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:36.555513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.588568+0000","last_change":"2024-05-08T21:53:23.586364+0000","last_active":"2024-05-08T21:53:23.588568+0000","last_peered":"2024-05-08T21:53:23.588568+0000","last_clean":"2024-05-08T21:53:23.588568+0000","last_became_active":"2024-05-08T21:53:23.585420+0000","last_became_peered":"2024-05-08T21:53:23.585420+0000","last_unstale":"2024-05-08T21:53:23.588568+0000","last_undegraded":"2024-05-08T21:53:23.588568+0000","last_fullsized":"2024-05-08T21:53:23.588568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:01.545821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:01.020694+0000","last_change":"2024-05-08T21:53:23.148809+0000","last_active":"2024-05-08T21:54:01.020694+0000","last_peered":"2024-05-08T21:54:01.020694+0000","last_clean":"2024-05-08T21:54:01.020694+0000","last_became_active":"2024-05-08T21:53:23.148532+0000","last_became_peered":"2024-05-08T21:53:23.148532+0000","last_unstale":"2024-05-08T21:54:01.020694+0000","last_undegraded":"2024-05-08T21:54:01.020694+0000","last_fullsized":"2024-05-08T21:54:01.020694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:48.209778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566739+0000","last_change":"2024-05-08T21:53:23.584120+0000","last_active":"2024-05-08T21:53:24.566739+0000","last_peered":"2024-05-08T21:53:24.566739+0000","last_clean":"2024-05-08T21:53:24.566739+0000","last_became_active":"2024-05-08T21:53:23.583804+0000","last_became_peered":"2024-05-08T21:53:23.583804+0000","last_unstale":"2024-05-08T21:53:24.566739+0000","last_undegraded":"2024-05-08T21:53:24.566739+0000","last_fullsized":"2024-05-08T21:53:24.566739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:10.669322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.196151+0000","last_change":"2024-05-08T21:53:23.144732+0000","last_active":"2024-05-08T21:53:26.196151+0000","last_peered":"2024-05-08T21:53:26.196151+0000","last_clean":"2024-05-08T21:53:26.196151+0000","last_became_active":"2024-05-08T21:53:23.144405+0000","last_became_peered":"2024-05-08T21:53:23.144405+0000","last_unstale":"2024-05-08T21:53:26.196151+0000","last_undegraded":"2024-05-08T21:53:26.196151+0000","last_fullsized":"2024-05-08T21:53:26.196151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:46.356499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780046+0000","last_change":"2024-05-08T21:53:23.578621+0000","last_active":"2024-05-08T21:53:23.780046+0000","last_peered":"2024-05-08T21:53:23.780046+0000","last_clean":"2024-05-08T21:53:23.780046+0000","last_became_active":"2024-05-08T21:53:23.578331+0000","last_became_peered":"2024-05-08T21:53:23.578331+0000","last_unstale":"2024-05-08T21:53:23.780046+0000","last_undegraded":"2024-05-08T21:53:23.780046+0000","last_fullsized":"2024-05-08T21:53:23.780046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:27.409751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:29.969142+0000","last_change":"2024-05-08T21:53:23.145398+0000","last_active":"2024-05-08T21:53:29.969142+0000","last_peered":"2024-05-08T21:53:29.969142+0000","last_clean":"2024-05-08T21:53:29.969142+0000","last_became_active":"2024-05-08T21:53:23.144997+0000","last_became_peered":"2024-05-08T21:53:23.144997+0000","last_unstale":"2024-05-08T21:53:29.969142+0000","last_undegraded":"2024-05-08T21:53:29.969142+0000","last_fullsized":"2024-05-08T21:53:29.969142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.116791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568566+0000","last_change":"2024-05-08T21:53:23.573118+0000","last_active":"2024-05-08T21:53:24.568566+0000","last_peered":"2024-05-08T21:53:24.568566+0000","last_clean":"2024-05-08T21:53:24.568566+0000","last_became_active":"2024-05-08T21:53:23.572639+0000","last_became_peered":"2024-05-08T21:53:23.572639+0000","last_unstale":"2024-05-08T21:53:24.568566+0000","last_undegraded":"2024-05-08T21:53:24.568566+0000","last_fullsized":"2024-05-08T21:53:24.568566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.786457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.964190+0000","last_change":"2024-05-08T21:53:23.144706+0000","last_active":"2024-05-08T21:53:28.964190+0000","last_peered":"2024-05-08T21:53:28.964190+0000","last_clean":"2024-05-08T21:53:28.964190+0000","last_became_active":"2024-05-08T21:53:23.144380+0000","last_became_peered":"2024-05-08T21:53:23.144380+0000","last_unstale":"2024-05-08T21:53:28.964190+0000","last_undegraded":"2024-05-08T21:53:28.964190+0000","last_fullsized":"2024-05-08T21:53:28.964190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:05.594755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832631+0000","last_change":"2024-05-08T21:53:23.573836+0000","last_active":"2024-05-08T21:53:24.832631+0000","last_peered":"2024-05-08T21:53:24.832631+0000","last_clean":"2024-05-08T21:53:24.832631+0000","last_became_active":"2024-05-08T21:53:23.572323+0000","last_became_peered":"2024-05-08T21:53:23.572323+0000","last_unstale":"2024-05-08T21:53:24.832631+0000","last_undegraded":"2024-05-08T21:53:24.832631+0000","last_fullsized":"2024-05-08T21:53:24.832631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:34.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:03.001941+0000","last_change":"2024-05-08T21:53:23.147919+0000","last_active":"2024-05-08T21:54:03.001941+0000","last_peered":"2024-05-08T21:54:03.001941+0000","last_clean":"2024-05-08T21:54:03.001941+0000","last_became_active":"2024-05-08T21:53:23.147473+0000","last_became_peered":"2024-05-08T21:53:23.147473+0000","last_unstale":"2024-05-08T21:54:03.001941+0000","last_undegraded":"2024-05-08T21:54:03.001941+0000","last_fullsized":"2024-05-08T21:54:03.001941+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:07.158398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566928+0000","last_change":"2024-05-08T21:53:23.576219+0000","last_active":"2024-05-08T21:53:24.566928+0000","last_peered":"2024-05-08T21:53:24.566928+0000","last_clean":"2024-05-08T21:53:24.566928+0000","last_became_active":"2024-05-08T21:53:23.575779+0000","last_became_peered":"2024-05-08T21:53:23.575779+0000","last_unstale":"2024-05-08T21:53:24.566928+0000","last_undegraded":"2024-05-08T21:53:24.566928+0000","last_fullsized":"2024-05-08T21:53:24.566928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:19.813711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953935+0000","last_change":"2024-05-08T21:53:23.152159+0000","last_active":"2024-05-08T21:53:27.953935+0000","last_peered":"2024-05-08T21:53:27.953935+0000","last_clean":"2024-05-08T21:53:27.953935+0000","last_became_active":"2024-05-08T21:53:23.151816+0000","last_became_peered":"2024-05-08T21:53:23.151816+0000","last_unstale":"2024-05-08T21:53:27.953935+0000","last_undegraded":"2024-05-08T21:53:27.953935+0000","last_fullsized":"2024-05-08T21:53:27.953935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.995097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832701+0000","last_change":"2024-05-08T21:53:23.575509+0000","last_active":"2024-05-08T21:53:24.832701+0000","last_peered":"2024-05-08T21:53:24.832701+0000","last_clean":"2024-05-08T21:53:24.832701+0000","last_became_active":"2024-05-08T21:53:23.574966+0000","last_became_peered":"2024-05-08T21:53:23.574966+0000","last_unstale":"2024-05-08T21:53:24.832701+0000","last_undegraded":"2024-05-08T21:53:24.832701+0000","last_fullsized":"2024-05-08T21:53:24.832701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:50.075249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.601006+0000","last_change":"2024-05-08T21:53:23.143151+0000","last_active":"2024-05-08T21:53:27.601006+0000","last_peered":"2024-05-08T21:53:27.601006+0000","last_clean":"2024-05-08T21:53:27.601006+0000","last_became_active":"2024-05-08T21:53:23.142829+0000","last_became_peered":"2024-05-08T21:53:23.142829+0000","last_unstale":"2024-05-08T21:53:27.601006+0000","last_undegraded":"2024-05-08T21:53:27.601006+0000","last_fullsized":"2024-05-08T21:53:27.601006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:49.372293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566098+0000","last_change":"2024-05-08T21:53:23.573587+0000","last_active":"2024-05-08T21:53:24.566098+0000","last_peered":"2024-05-08T21:53:24.566098+0000","last_clean":"2024-05-08T21:53:24.566098+0000","last_became_active":"2024-05-08T21:53:23.573329+0000","last_became_peered":"2024-05-08T21:53:23.573329+0000","last_unstale":"2024-05-08T21:53:24.566098+0000","last_undegraded":"2024-05-08T21:53:24.566098+0000","last_fullsized":"2024-05-08T21:53:24.566098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:49.915501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963958+0000","last_change":"2024-05-08T21:53:23.144147+0000","last_active":"2024-05-08T21:53:28.963958+0000","last_peered":"2024-05-08T21:53:28.963958+0000","last_clean":"2024-05-08T21:53:28.963958+0000","last_became_active":"2024-05-08T21:53:23.143808+0000","last_became_peered":"2024-05-08T21:53:23.143808+0000","last_unstale":"2024-05-08T21:53:28.963958+0000","last_undegraded":"2024-05-08T21:53:28.963958+0000","last_fullsized":"2024-05-08T21:53:28.963958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:07.011887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.592090+0000","last_change":"2024-05-08T21:53:23.583193+0000","last_active":"2024-05-08T21:53:26.592090+0000","last_peered":"2024-05-08T21:53:26.592090+0000","last_clean":"2024-05-08T21:53:26.592090+0000","last_became_active":"2024-05-08T21:53:23.582833+0000","last_became_peered":"2024-05-08T21:53:23.582833+0000","last_unstale":"2024-05-08T21:53:26.592090+0000","last_undegraded":"2024-05-08T21:53:26.592090+0000","last_fullsized":"2024-05-08T21:53:26.592090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:08.708225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":90,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:05.003916+0000","last_change":"2024-05-08T21:53:23.146398+0000","last_active":"2024-05-08T21:54:05.003916+0000","last_peered":"2024-05-08T21:54:05.003916+0000","last_clean":"2024-05-08T21:54:05.003916+0000","last_became_active":"2024-05-08T21:53:23.145823+0000","last_became_peered":"2024-05-08T21:53:23.145823+0000","last_unstale":"2024-05-08T21:54:05.003916+0000","last_undegraded":"2024-05-08T21:54:05.003916+0000","last_fullsized":"2024-05-08T21:54:05.003916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.535292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568901+0000","last_change":"2024-05-08T21:53:23.578664+0000","last_active":"2024-05-08T21:53:24.568901+0000","last_peered":"2024-05-08T21:53:24.568901+0000","last_clean":"2024-05-08T21:53:24.568901+0000","last_became_active":"2024-05-08T21:53:23.578398+0000","last_became_peered":"2024-05-08T21:53:23.578398+0000","last_unstale":"2024-05-08T21:53:24.568901+0000","last_undegraded":"2024-05-08T21:53:24.568901+0000","last_fullsized":"2024-05-08T21:53:24.568901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:10.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.961453+0000","last_change":"2024-05-08T21:53:23.151071+0000","last_active":"2024-05-08T21:53:28.961453+0000","last_peered":"2024-05-08T21:53:28.961453+0000","last_clean":"2024-05-08T21:53:28.961453+0000","last_became_active":"2024-05-08T21:53:23.150566+0000","last_became_peered":"2024-05-08T21:53:23.150566+0000","last_unstale":"2024-05-08T21:53:28.961453+0000","last_undegraded":"2024-05-08T21:53:28.961453+0000","last_fullsized":"2024-05-08T21:53:28.961453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:51.845419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.780026+0000","last_change":"2024-05-08T21:53:23.573421+0000","last_active":"2024-05-08T21:53:23.780026+0000","last_peered":"2024-05-08T21:53:23.780026+0000","last_clean":"2024-05-08T21:53:23.780026+0000","last_became_active":"2024-05-08T21:53:23.571240+0000","last_became_peered":"2024-05-08T21:53:23.571240+0000","last_unstale":"2024-05-08T21:53:23.780026+0000","last_undegraded":"2024-05-08T21:53:23.780026+0000","last_fullsized":"2024-05-08T21:53:23.780026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:51.695322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.947244+0000","last_change":"2024-05-08T21:53:23.144552+0000","last_active":"2024-05-08T21:53:27.947244+0000","last_peered":"2024-05-08T21:53:27.947244+0000","last_clean":"2024-05-08T21:53:27.947244+0000","last_became_active":"2024-05-08T21:53:23.144231+0000","last_became_peered":"2024-05-08T21:53:23.144231+0000","last_unstale":"2024-05-08T21:53:27.947244+0000","last_undegraded":"2024-05-08T21:53:27.947244+0000","last_fullsized":"2024-05-08T21:53:27.947244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:44.877035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568575+0000","last_change":"2024-05-08T21:53:23.572913+0000","last_active":"2024-05-08T21:53:24.568575+0000","last_peered":"2024-05-08T21:53:24.568575+0000","last_clean":"2024-05-08T21:53:24.568575+0000","last_became_active":"2024-05-08T21:53:23.572256+0000","last_became_peered":"2024-05-08T21:53:23.572256+0000","last_unstale":"2024-05-08T21:53:24.568575+0000","last_undegraded":"2024-05-08T21:53:24.568575+0000","last_fullsized":"2024-05-08T21:53:24.568575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:05.485571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'30","reported_seq":127,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:05.015456+0000","last_change":"2024-05-08T21:53:23.150978+0000","last_active":"2024-05-08T21:54:05.015456+0000","last_peered":"2024-05-08T21:54:05.015456+0000","last_clean":"2024-05-08T21:54:05.015456+0000","last_became_active":"2024-05-08T21:53:23.150637+0000","last_became_peered":"2024-05-08T21:53:23.150637+0000","last_unstale":"2024-05-08T21:54:05.015456+0000","last_undegraded":"2024-05-08T21:54:05.015456+0000","last_fullsized":"2024-05-08T21:54:05.015456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:40.569414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61000,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":102,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566921+0000","last_change":"2024-05-08T21:53:23.581145+0000","last_active":"2024-05-08T21:53:24.566921+0000","last_peered":"2024-05-08T21:53:24.566921+0000","last_clean":"2024-05-08T21:53:24.566921+0000","last_became_active":"2024-05-08T21:53:23.580727+0000","last_became_peered":"2024-05-08T21:53:23.580727+0000","last_unstale":"2024-05-08T21:53:24.566921+0000","last_undegraded":"2024-05-08T21:53:24.566921+0000","last_fullsized":"2024-05-08T21:53:24.566921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:58:21.737504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.962849+0000","last_change":"2024-05-08T21:53:23.146278+0000","last_active":"2024-05-08T21:53:28.962849+0000","last_peered":"2024-05-08T21:53:28.962849+0000","last_clean":"2024-05-08T21:53:28.962849+0000","last_became_active":"2024-05-08T21:53:23.145965+0000","last_became_peered":"2024-05-08T21:53:23.145965+0000","last_unstale":"2024-05-08T21:53:28.962849+0000","last_undegraded":"2024-05-08T21:53:28.962849+0000","last_fullsized":"2024-05-08T21:53:28.962849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:05.370847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806557+0000","last_change":"2024-05-08T21:53:23.576732+0000","last_active":"2024-05-08T21:53:23.806557+0000","last_peered":"2024-05-08T21:53:23.806557+0000","last_clean":"2024-05-08T21:53:23.806557+0000","last_became_active":"2024-05-08T21:53:23.576415+0000","last_became_peered":"2024-05-08T21:53:23.576415+0000","last_unstale":"2024-05-08T21:53:23.806557+0000","last_undegraded":"2024-05-08T21:53:23.806557+0000","last_fullsized":"2024-05-08T21:53:23.806557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:47.184028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963718+0000","last_change":"2024-05-08T21:53:23.148398+0000","last_active":"2024-05-08T21:53:28.963718+0000","last_peered":"2024-05-08T21:53:28.963718+0000","last_clean":"2024-05-08T21:53:28.963718+0000","last_became_active":"2024-05-08T21:53:23.148078+0000","last_became_peered":"2024-05-08T21:53:23.148078+0000","last_unstale":"2024-05-08T21:53:28.963718+0000","last_undegraded":"2024-05-08T21:53:28.963718+0000","last_fullsized":"2024-05-08T21:53:28.963718+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:38.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889064+0000","last_change":"2024-05-08T21:53:23.581700+0000","last_active":"2024-05-08T21:53:25.889064+0000","last_peered":"2024-05-08T21:53:25.889064+0000","last_clean":"2024-05-08T21:53:25.889064+0000","last_became_active":"2024-05-08T21:53:23.581346+0000","last_became_peered":"2024-05-08T21:53:23.581346+0000","last_unstale":"2024-05-08T21:53:25.889064+0000","last_undegraded":"2024-05-08T21:53:25.889064+0000","last_fullsized":"2024-05-08T21:53:25.889064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:08.530505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":57,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:01.999697+0000","last_change":"2024-05-08T21:53:23.148415+0000","last_active":"2024-05-08T21:54:01.999697+0000","last_peered":"2024-05-08T21:54:01.999697+0000","last_clean":"2024-05-08T21:54:01.999697+0000","last_became_active":"2024-05-08T21:53:23.148037+0000","last_became_peered":"2024-05-08T21:53:23.148037+0000","last_unstale":"2024-05-08T21:54:01.999697+0000","last_undegraded":"2024-05-08T21:54:01.999697+0000","last_fullsized":"2024-05-08T21:54:01.999697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:17.367009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.888950+0000","last_change":"2024-05-08T21:53:23.589596+0000","last_active":"2024-05-08T21:53:25.888950+0000","last_peered":"2024-05-08T21:53:25.888950+0000","last_clean":"2024-05-08T21:53:25.888950+0000","last_became_active":"2024-05-08T21:53:23.589309+0000","last_became_peered":"2024-05-08T21:53:23.589309+0000","last_unstale":"2024-05-08T21:53:25.888950+0000","last_undegraded":"2024-05-08T21:53:25.888950+0000","last_fullsized":"2024-05-08T21:53:25.888950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:41.814532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954546+0000","last_change":"2024-05-08T21:53:23.145434+0000","last_active":"2024-05-08T21:53:27.954546+0000","last_peered":"2024-05-08T21:53:27.954546+0000","last_clean":"2024-05-08T21:53:27.954546+0000","last_became_active":"2024-05-08T21:53:23.144328+0000","last_became_peered":"2024-05-08T21:53:23.144328+0000","last_unstale":"2024-05-08T21:53:27.954546+0000","last_undegraded":"2024-05-08T21:53:27.954546+0000","last_fullsized":"2024-05-08T21:53:27.954546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:30.389806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832474+0000","last_change":"2024-05-08T21:53:23.572862+0000","last_active":"2024-05-08T21:53:24.832474+0000","last_peered":"2024-05-08T21:53:24.832474+0000","last_clean":"2024-05-08T21:53:24.832474+0000","last_became_active":"2024-05-08T21:53:23.572516+0000","last_became_peered":"2024-05-08T21:53:23.572516+0000","last_unstale":"2024-05-08T21:53:24.832474+0000","last_undegraded":"2024-05-08T21:53:24.832474+0000","last_fullsized":"2024-05-08T21:53:24.832474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:37.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.944692+0000","last_change":"2024-05-08T21:53:23.146282+0000","last_active":"2024-05-08T21:53:27.944692+0000","last_peered":"2024-05-08T21:53:27.944692+0000","last_clean":"2024-05-08T21:53:27.944692+0000","last_became_active":"2024-05-08T21:53:23.145974+0000","last_became_peered":"2024-05-08T21:53:23.145974+0000","last_unstale":"2024-05-08T21:53:27.944692+0000","last_undegraded":"2024-05-08T21:53:27.944692+0000","last_fullsized":"2024-05-08T21:53:27.944692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:26.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703149+0000","last_change":"2024-05-08T21:53:23.578164+0000","last_active":"2024-05-08T21:53:23.703149+0000","last_peered":"2024-05-08T21:53:23.703149+0000","last_clean":"2024-05-08T21:53:23.703149+0000","last_became_active":"2024-05-08T21:53:23.577832+0000","last_became_peered":"2024-05-08T21:53:23.577832+0000","last_unstale":"2024-05-08T21:53:23.703149+0000","last_undegraded":"2024-05-08T21:53:23.703149+0000","last_fullsized":"2024-05-08T21:53:23.703149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:18.700875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.592388+0000","last_change":"2024-05-08T21:53:23.151368+0000","last_active":"2024-05-08T21:53:26.592388+0000","last_peered":"2024-05-08T21:53:26.592388+0000","last_clean":"2024-05-08T21:53:26.592388+0000","last_became_active":"2024-05-08T21:53:23.151068+0000","last_became_peered":"2024-05-08T21:53:23.151068+0000","last_unstale":"2024-05-08T21:53:26.592388+0000","last_undegraded":"2024-05-08T21:53:26.592388+0000","last_fullsized":"2024-05-08T21:53:26.592388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:42.662149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.275007+0000","last_change":"2024-05-08T21:53:23.581387+0000","last_active":"2024-05-08T21:53:25.275007+0000","last_peered":"2024-05-08T21:53:25.275007+0000","last_clean":"2024-05-08T21:53:25.275007+0000","last_became_active":"2024-05-08T21:53:23.579346+0000","last_became_peered":"2024-05-08T21:53:23.579346+0000","last_unstale":"2024-05-08T21:53:25.275007+0000","last_undegraded":"2024-05-08T21:53:25.275007+0000","last_fullsized":"2024-05-08T21:53:25.275007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:42.997082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":55,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:04.025529+0000","last_change":"2024-05-08T21:53:23.150218+0000","last_active":"2024-05-08T21:54:04.025529+0000","last_peered":"2024-05-08T21:54:04.025529+0000","last_clean":"2024-05-08T21:54:04.025529+0000","last_became_active":"2024-05-08T21:53:23.148582+0000","last_became_peered":"2024-05-08T21:53:23.148582+0000","last_unstale":"2024-05-08T21:54:04.025529+0000","last_undegraded":"2024-05-08T21:54:04.025529+0000","last_fullsized":"2024-05-08T21:54:04.025529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:32.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.113790+0000","last_change":"2024-05-08T21:53:23.579971+0000","last_active":"2024-05-08T21:53:25.113790+0000","last_peered":"2024-05-08T21:53:25.113790+0000","last_clean":"2024-05-08T21:53:25.113790+0000","last_became_active":"2024-05-08T21:53:23.578267+0000","last_became_peered":"2024-05-08T21:53:23.578267+0000","last_unstale":"2024-05-08T21:53:25.113790+0000","last_undegraded":"2024-05-08T21:53:25.113790+0000","last_fullsized":"2024-05-08T21:53:25.113790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:20.650914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832545+0000","last_change":"2024-05-08T21:53:23.584126+0000","last_active":"2024-05-08T21:53:24.832545+0000","last_peered":"2024-05-08T21:53:24.832545+0000","last_clean":"2024-05-08T21:53:24.832545+0000","last_became_active":"2024-05-08T21:53:23.583801+0000","last_became_peered":"2024-05-08T21:53:23.583801+0000","last_unstale":"2024-05-08T21:53:24.832545+0000","last_undegraded":"2024-05-08T21:53:24.832545+0000","last_fullsized":"2024-05-08T21:53:24.832545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:55:19.487948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":54,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:01.999087+0000","last_change":"2024-05-08T21:53:23.145221+0000","last_active":"2024-05-08T21:54:01.999087+0000","last_peered":"2024-05-08T21:54:01.999087+0000","last_clean":"2024-05-08T21:54:01.999087+0000","last_became_active":"2024-05-08T21:53:23.144659+0000","last_became_peered":"2024-05-08T21:53:23.144659+0000","last_unstale":"2024-05-08T21:54:01.999087+0000","last_undegraded":"2024-05-08T21:54:01.999087+0000","last_fullsized":"2024-05-08T21:54:01.999087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:58.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566506+0000","last_change":"2024-05-08T21:53:23.581748+0000","last_active":"2024-05-08T21:53:24.566506+0000","last_peered":"2024-05-08T21:53:24.566506+0000","last_clean":"2024-05-08T21:53:24.566506+0000","last_became_active":"2024-05-08T21:53:23.581336+0000","last_became_peered":"2024-05-08T21:53:23.581336+0000","last_unstale":"2024-05-08T21:53:24.566506+0000","last_undegraded":"2024-05-08T21:53:24.566506+0000","last_fullsized":"2024-05-08T21:53:24.566506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:31.158473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963098+0000","last_change":"2024-05-08T21:53:23.147073+0000","last_active":"2024-05-08T21:53:28.963098+0000","last_peered":"2024-05-08T21:53:28.963098+0000","last_clean":"2024-05-08T21:53:28.963098+0000","last_became_active":"2024-05-08T21:53:23.146720+0000","last_became_peered":"2024-05-08T21:53:23.146720+0000","last_unstale":"2024-05-08T21:53:28.963098+0000","last_undegraded":"2024-05-08T21:53:28.963098+0000","last_fullsized":"2024-05-08T21:53:28.963098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:40.566772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.977449+0000","last_change":"2024-05-08T21:52:33.304674+0000","last_active":"2024-05-08T21:53:25.977449+0000","last_peered":"2024-05-08T21:53:25.977449+0000","last_clean":"2024-05-08T21:53:25.977449+0000","last_became_active":"2024-05-08T21:52:33.304193+0000","last_became_peered":"2024-05-08T21:52:33.304193+0000","last_unstale":"2024-05-08T21:53:25.977449+0000","last_undegraded":"2024-05-08T21:53:25.977449+0000","last_fullsized":"2024-05-08T21:53:25.977449+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:50:08.558194+0000","last_clean_scrub_stamp":"2024-05-08T21:50:08.558194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:21.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806496+0000","last_change":"2024-05-08T21:53:23.583793+0000","last_active":"2024-05-08T21:53:23.806496+0000","last_peered":"2024-05-08T21:53:23.806496+0000","last_clean":"2024-05-08T21:53:23.806496+0000","last_became_active":"2024-05-08T21:53:23.583464+0000","last_became_peered":"2024-05-08T21:53:23.583464+0000","last_unstale":"2024-05-08T21:53:23.806496+0000","last_undegraded":"2024-05-08T21:53:23.806496+0000","last_fullsized":"2024-05-08T21:53:23.806496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:02.106595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.600923+0000","last_change":"2024-05-08T21:53:23.150460+0000","last_active":"2024-05-08T21:53:27.600923+0000","last_peered":"2024-05-08T21:53:27.600923+0000","last_clean":"2024-05-08T21:53:27.600923+0000","last_became_active":"2024-05-08T21:53:23.150025+0000","last_became_peered":"2024-05-08T21:53:23.150025+0000","last_unstale":"2024-05-08T21:53:27.600923+0000","last_undegraded":"2024-05-08T21:53:27.600923+0000","last_fullsized":"2024-05-08T21:53:27.600923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:28:09.311683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.954963+0000","last_change":"2024-05-08T21:53:23.143160+0000","last_active":"2024-05-08T21:53:27.954963+0000","last_peered":"2024-05-08T21:53:27.954963+0000","last_clean":"2024-05-08T21:53:27.954963+0000","last_became_active":"2024-05-08T21:53:23.142718+0000","last_became_peered":"2024-05-08T21:53:23.142718+0000","last_unstale":"2024-05-08T21:53:27.954963+0000","last_undegraded":"2024-05-08T21:53:27.954963+0000","last_fullsized":"2024-05-08T21:53:27.954963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:54.148427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806605+0000","last_change":"2024-05-08T21:53:23.587069+0000","last_active":"2024-05-08T21:53:23.806605+0000","last_peered":"2024-05-08T21:53:23.806605+0000","last_clean":"2024-05-08T21:53:23.806605+0000","last_became_active":"2024-05-08T21:53:23.586782+0000","last_became_peered":"2024-05-08T21:53:23.586782+0000","last_unstale":"2024-05-08T21:53:23.806605+0000","last_undegraded":"2024-05-08T21:53:23.806605+0000","last_fullsized":"2024-05-08T21:53:23.806605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:08.897700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:32.438589+0000","last_change":"2024-05-08T21:53:23.142641+0000","last_active":"2024-05-08T21:53:32.438589+0000","last_peered":"2024-05-08T21:53:32.438589+0000","last_clean":"2024-05-08T21:53:32.438589+0000","last_became_active":"2024-05-08T21:53:23.142294+0000","last_became_peered":"2024-05-08T21:53:23.142294+0000","last_unstale":"2024-05-08T21:53:32.438589+0000","last_undegraded":"2024-05-08T21:53:32.438589+0000","last_fullsized":"2024-05-08T21:53:32.438589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:11.687406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566396+0000","last_change":"2024-05-08T21:53:23.580729+0000","last_active":"2024-05-08T21:53:24.566396+0000","last_peered":"2024-05-08T21:53:24.566396+0000","last_clean":"2024-05-08T21:53:24.566396+0000","last_became_active":"2024-05-08T21:53:23.580159+0000","last_became_peered":"2024-05-08T21:53:23.580159+0000","last_unstale":"2024-05-08T21:53:24.566396+0000","last_undegraded":"2024-05-08T21:53:24.566396+0000","last_fullsized":"2024-05-08T21:53:24.566396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:43:33.336308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963066+0000","last_change":"2024-05-08T21:53:23.144242+0000","last_active":"2024-05-08T21:53:28.963066+0000","last_peered":"2024-05-08T21:53:28.963066+0000","last_clean":"2024-05-08T21:53:28.963066+0000","last_became_active":"2024-05-08T21:53:23.143905+0000","last_became_peered":"2024-05-08T21:53:23.143905+0000","last_unstale":"2024-05-08T21:53:28.963066+0000","last_undegraded":"2024-05-08T21:53:28.963066+0000","last_fullsized":"2024-05-08T21:53:28.963066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:41.990942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779991+0000","last_change":"2024-05-08T21:53:23.576972+0000","last_active":"2024-05-08T21:53:23.779991+0000","last_peered":"2024-05-08T21:53:23.779991+0000","last_clean":"2024-05-08T21:53:23.779991+0000","last_became_active":"2024-05-08T21:53:23.576668+0000","last_became_peered":"2024-05-08T21:53:23.576668+0000","last_unstale":"2024-05-08T21:53:23.779991+0000","last_undegraded":"2024-05-08T21:53:23.779991+0000","last_fullsized":"2024-05-08T21:53:23.779991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:25.545108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.595186+0000","last_change":"2024-05-08T21:53:23.145610+0000","last_active":"2024-05-08T21:53:26.595186+0000","last_peered":"2024-05-08T21:53:26.595186+0000","last_clean":"2024-05-08T21:53:26.595186+0000","last_became_active":"2024-05-08T21:53:23.145075+0000","last_became_peered":"2024-05-08T21:53:23.145075+0000","last_unstale":"2024-05-08T21:53:26.595186+0000","last_undegraded":"2024-05-08T21:53:26.595186+0000","last_fullsized":"2024-05-08T21:53:26.595186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:58.351303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566650+0000","last_change":"2024-05-08T21:53:23.579335+0000","last_active":"2024-05-08T21:53:24.566650+0000","last_peered":"2024-05-08T21:53:24.566650+0000","last_clean":"2024-05-08T21:53:24.566650+0000","last_became_active":"2024-05-08T21:53:23.578956+0000","last_became_peered":"2024-05-08T21:53:23.578956+0000","last_unstale":"2024-05-08T21:53:24.566650+0000","last_undegraded":"2024-05-08T21:53:24.566650+0000","last_fullsized":"2024-05-08T21:53:24.566650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:34.872954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.964020+0000","last_change":"2024-05-08T21:53:23.144108+0000","last_active":"2024-05-08T21:53:28.964020+0000","last_peered":"2024-05-08T21:53:28.964020+0000","last_clean":"2024-05-08T21:53:28.964020+0000","last_became_active":"2024-05-08T21:53:23.143786+0000","last_became_peered":"2024-05-08T21:53:23.143786+0000","last_unstale":"2024-05-08T21:53:28.964020+0000","last_undegraded":"2024-05-08T21:53:28.964020+0000","last_fullsized":"2024-05-08T21:53:28.964020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:15.512879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703096+0000","last_change":"2024-05-08T21:53:23.587667+0000","last_active":"2024-05-08T21:53:23.703096+0000","last_peered":"2024-05-08T21:53:23.703096+0000","last_clean":"2024-05-08T21:53:23.703096+0000","last_became_active":"2024-05-08T21:53:23.586908+0000","last_became_peered":"2024-05-08T21:53:23.586908+0000","last_unstale":"2024-05-08T21:53:23.703096+0000","last_undegraded":"2024-05-08T21:53:23.703096+0000","last_fullsized":"2024-05-08T21:53:23.703096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:06.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:32.081481+0000","last_change":"2024-05-08T21:53:23.150602+0000","last_active":"2024-05-08T21:53:32.081481+0000","last_peered":"2024-05-08T21:53:32.081481+0000","last_clean":"2024-05-08T21:53:32.081481+0000","last_became_active":"2024-05-08T21:53:23.150256+0000","last_became_peered":"2024-05-08T21:53:23.150256+0000","last_unstale":"2024-05-08T21:53:32.081481+0000","last_undegraded":"2024-05-08T21:53:32.081481+0000","last_fullsized":"2024-05-08T21:53:32.081481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:22:05.088246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566829+0000","last_change":"2024-05-08T21:53:23.577768+0000","last_active":"2024-05-08T21:53:24.566829+0000","last_peered":"2024-05-08T21:53:24.566829+0000","last_clean":"2024-05-08T21:53:24.566829+0000","last_became_active":"2024-05-08T21:53:23.577388+0000","last_became_peered":"2024-05-08T21:53:23.577388+0000","last_unstale":"2024-05-08T21:53:24.566829+0000","last_undegraded":"2024-05-08T21:53:24.566829+0000","last_fullsized":"2024-05-08T21:53:24.566829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:24.707904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:26.593957+0000","last_change":"2024-05-08T21:53:23.145165+0000","last_active":"2024-05-08T21:53:26.593957+0000","last_peered":"2024-05-08T21:53:26.593957+0000","last_clean":"2024-05-08T21:53:26.593957+0000","last_became_active":"2024-05-08T21:53:23.144465+0000","last_became_peered":"2024-05-08T21:53:23.144465+0000","last_unstale":"2024-05-08T21:53:26.593957+0000","last_undegraded":"2024-05-08T21:53:26.593957+0000","last_fullsized":"2024-05-08T21:53:26.593957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:47.229905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889021+0000","last_change":"2024-05-08T21:53:23.583476+0000","last_active":"2024-05-08T21:53:25.889021+0000","last_peered":"2024-05-08T21:53:25.889021+0000","last_clean":"2024-05-08T21:53:25.889021+0000","last_became_active":"2024-05-08T21:53:23.582989+0000","last_became_peered":"2024-05-08T21:53:23.582989+0000","last_unstale":"2024-05-08T21:53:25.889021+0000","last_undegraded":"2024-05-08T21:53:25.889021+0000","last_fullsized":"2024-05-08T21:53:25.889021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:05.402755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":53,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:54:02.034414+0000","last_change":"2024-05-08T21:53:23.147469+0000","last_active":"2024-05-08T21:54:02.034414+0000","last_peered":"2024-05-08T21:54:02.034414+0000","last_clean":"2024-05-08T21:54:02.034414+0000","last_became_active":"2024-05-08T21:53:23.147145+0000","last_became_peered":"2024-05-08T21:53:23.147145+0000","last_unstale":"2024-05-08T21:54:02.034414+0000","last_undegraded":"2024-05-08T21:54:02.034414+0000","last_fullsized":"2024-05-08T21:54:02.034414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:15.806118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889101+0000","last_change":"2024-05-08T21:53:23.583221+0000","last_active":"2024-05-08T21:53:25.889101+0000","last_peered":"2024-05-08T21:53:25.889101+0000","last_clean":"2024-05-08T21:53:25.889101+0000","last_became_active":"2024-05-08T21:53:23.582942+0000","last_became_peered":"2024-05-08T21:53:23.582942+0000","last_unstale":"2024-05-08T21:53:25.889101+0000","last_undegraded":"2024-05-08T21:53:25.889101+0000","last_fullsized":"2024-05-08T21:53:25.889101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:32.188611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:27.953180+0000","last_change":"2024-05-08T21:53:23.143721+0000","last_active":"2024-05-08T21:53:27.953180+0000","last_peered":"2024-05-08T21:53:27.953180+0000","last_clean":"2024-05-08T21:53:27.953180+0000","last_became_active":"2024-05-08T21:53:23.143227+0000","last_became_peered":"2024-05-08T21:53:23.143227+0000","last_unstale":"2024-05-08T21:53:27.953180+0000","last_undegraded":"2024-05-08T21:53:27.953180+0000","last_fullsized":"2024-05-08T21:53:27.953180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:09.619865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.977682+0000","last_change":"2024-05-08T21:53:23.585820+0000","last_active":"2024-05-08T21:53:25.977682+0000","last_peered":"2024-05-08T21:53:25.977682+0000","last_clean":"2024-05-08T21:53:25.977682+0000","last_became_active":"2024-05-08T21:53:23.585435+0000","last_became_peered":"2024-05-08T21:53:23.585435+0000","last_unstale":"2024-05-08T21:53:25.977682+0000","last_undegraded":"2024-05-08T21:53:25.977682+0000","last_fullsized":"2024-05-08T21:53:25.977682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:48.696651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:28.963041+0000","last_change":"2024-05-08T21:53:23.141992+0000","last_active":"2024-05-08T21:53:28.963041+0000","last_peered":"2024-05-08T21:53:28.963041+0000","last_clean":"2024-05-08T21:53:28.963041+0000","last_became_active":"2024-05-08T21:53:23.141670+0000","last_became_peered":"2024-05-08T21:53:23.141670+0000","last_unstale":"2024-05-08T21:53:28.963041+0000","last_undegraded":"2024-05-08T21:53:28.963041+0000","last_fullsized":"2024-05-08T21:53:28.963041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:22.113255+0000","last_clean_scrub_stamp":"2024-05-08T21:53:22.113255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:53.191681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:25.889139+0000","last_change":"2024-05-08T21:53:23.586649+0000","last_active":"2024-05-08T21:53:25.889139+0000","last_peered":"2024-05-08T21:53:25.889139+0000","last_clean":"2024-05-08T21:53:25.889139+0000","last_became_active":"2024-05-08T21:53:23.586069+0000","last_became_peered":"2024-05-08T21:53:23.586069+0000","last_unstale":"2024-05-08T21:53:25.889139+0000","last_undegraded":"2024-05-08T21:53:25.889139+0000","last_fullsized":"2024-05-08T21:53:25.889139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:59.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703112+0000","last_change":"2024-05-08T21:53:23.588268+0000","last_active":"2024-05-08T21:53:23.703112+0000","last_peered":"2024-05-08T21:53:23.703112+0000","last_clean":"2024-05-08T21:53:23.703112+0000","last_became_active":"2024-05-08T21:53:23.587858+0000","last_became_peered":"2024-05-08T21:53:23.587858+0000","last_unstale":"2024-05-08T21:53:23.703112+0000","last_undegraded":"2024-05-08T21:53:23.703112+0000","last_fullsized":"2024-05-08T21:53:23.703112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:47.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.589133+0000","last_change":"2024-05-08T21:53:23.587227+0000","last_active":"2024-05-08T21:53:23.589133+0000","last_peered":"2024-05-08T21:53:23.589133+0000","last_clean":"2024-05-08T21:53:23.589133+0000","last_became_active":"2024-05-08T21:53:23.586807+0000","last_became_peered":"2024-05-08T21:53:23.586807+0000","last_unstale":"2024-05-08T21:53:23.589133+0000","last_undegraded":"2024-05-08T21:53:23.589133+0000","last_fullsized":"2024-05-08T21:53:23.589133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:29.814057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585531+0000","last_change":"2024-05-08T21:53:23.583968+0000","last_active":"2024-05-08T21:53:23.585531+0000","last_peered":"2024-05-08T21:53:23.585531+0000","last_clean":"2024-05-08T21:53:23.585531+0000","last_became_active":"2024-05-08T21:53:23.583625+0000","last_became_peered":"2024-05-08T21:53:23.583625+0000","last_unstale":"2024-05-08T21:53:23.585531+0000","last_undegraded":"2024-05-08T21:53:23.585531+0000","last_fullsized":"2024-05-08T21:53:23.585531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:40.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.779949+0000","last_change":"2024-05-08T21:53:23.571284+0000","last_active":"2024-05-08T21:53:23.779949+0000","last_peered":"2024-05-08T21:53:23.779949+0000","last_clean":"2024-05-08T21:53:23.779949+0000","last_became_active":"2024-05-08T21:53:23.570915+0000","last_became_peered":"2024-05-08T21:53:23.570915+0000","last_unstale":"2024-05-08T21:53:23.779949+0000","last_undegraded":"2024-05-08T21:53:23.779949+0000","last_fullsized":"2024-05-08T21:53:23.779949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:52.562178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806616+0000","last_change":"2024-05-08T21:53:23.572011+0000","last_active":"2024-05-08T21:53:23.806616+0000","last_peered":"2024-05-08T21:53:23.806616+0000","last_clean":"2024-05-08T21:53:23.806616+0000","last_became_active":"2024-05-08T21:53:23.571788+0000","last_became_peered":"2024-05-08T21:53:23.571788+0000","last_unstale":"2024-05-08T21:53:23.806616+0000","last_undegraded":"2024-05-08T21:53:23.806616+0000","last_fullsized":"2024-05-08T21:53:23.806616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:43.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703137+0000","last_change":"2024-05-08T21:53:23.582823+0000","last_active":"2024-05-08T21:53:23.703137+0000","last_peered":"2024-05-08T21:53:23.703137+0000","last_clean":"2024-05-08T21:53:23.703137+0000","last_became_active":"2024-05-08T21:53:23.582366+0000","last_became_peered":"2024-05-08T21:53:23.582366+0000","last_unstale":"2024-05-08T21:53:23.703137+0000","last_undegraded":"2024-05-08T21:53:23.703137+0000","last_fullsized":"2024-05-08T21:53:23.703137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:44.029915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.585670+0000","last_change":"2024-05-08T21:53:23.583566+0000","last_active":"2024-05-08T21:53:23.585670+0000","last_peered":"2024-05-08T21:53:23.585670+0000","last_clean":"2024-05-08T21:53:23.585670+0000","last_became_active":"2024-05-08T21:53:23.583204+0000","last_became_peered":"2024-05-08T21:53:23.583204+0000","last_unstale":"2024-05-08T21:53:23.585670+0000","last_undegraded":"2024-05-08T21:53:23.585670+0000","last_fullsized":"2024-05-08T21:53:23.585670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:41.426005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566280+0000","last_change":"2024-05-08T21:53:23.587105+0000","last_active":"2024-05-08T21:53:24.566280+0000","last_peered":"2024-05-08T21:53:24.566280+0000","last_clean":"2024-05-08T21:53:24.566280+0000","last_became_active":"2024-05-08T21:53:23.586773+0000","last_became_peered":"2024-05-08T21:53:23.586773+0000","last_unstale":"2024-05-08T21:53:24.566280+0000","last_undegraded":"2024-05-08T21:53:24.566280+0000","last_fullsized":"2024-05-08T21:53:24.566280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:50.809186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.832696+0000","last_change":"2024-05-08T21:53:23.576779+0000","last_active":"2024-05-08T21:53:24.832696+0000","last_peered":"2024-05-08T21:53:24.832696+0000","last_clean":"2024-05-08T21:53:24.832696+0000","last_became_active":"2024-05-08T21:53:23.576434+0000","last_became_peered":"2024-05-08T21:53:23.576434+0000","last_unstale":"2024-05-08T21:53:24.832696+0000","last_undegraded":"2024-05-08T21:53:24.832696+0000","last_fullsized":"2024-05-08T21:53:24.832696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:10.168522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.566243+0000","last_change":"2024-05-08T21:53:23.585196+0000","last_active":"2024-05-08T21:53:24.566243+0000","last_peered":"2024-05-08T21:53:24.566243+0000","last_clean":"2024-05-08T21:53:24.566243+0000","last_became_active":"2024-05-08T21:53:23.584160+0000","last_became_peered":"2024-05-08T21:53:23.584160+0000","last_unstale":"2024-05-08T21:53:24.566243+0000","last_undegraded":"2024-05-08T21:53:24.566243+0000","last_fullsized":"2024-05-08T21:53:24.566243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:15.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T21:53:24.568545+0000","last_change":"2024-05-08T21:53:23.577887+0000","last_active":"2024-05-08T21:53:24.568545+0000","last_peered":"2024-05-08T21:53:24.568545+0000","last_clean":"2024-05-08T21:53:24.568545+0000","last_became_active":"2024-05-08T21:53:23.577597+0000","last_became_peered":"2024-05-08T21:53:23.577597+0000","last_unstale":"2024-05-08T21:53:24.568545+0000","last_undegraded":"2024-05-08T21:53:24.568545+0000","last_fullsized":"2024-05-08T21:53:24.568545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:33.704002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.806594+0000","last_change":"2024-05-08T21:53:23.575180+0000","last_active":"2024-05-08T21:53:23.806594+0000","last_peered":"2024-05-08T21:53:23.806594+0000","last_clean":"2024-05-08T21:53:23.806594+0000","last_became_active":"2024-05-08T21:53:23.574521+0000","last_became_peered":"2024-05-08T21:53:23.574521+0000","last_unstale":"2024-05-08T21:53:23.806594+0000","last_undegraded":"2024-05-08T21:53:23.806594+0000","last_fullsized":"2024-05-08T21:53:23.806594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:59.609492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T21:53:23.703125+0000","last_change":"2024-05-08T21:53:23.585380+0000","last_active":"2024-05-08T21:53:23.703125+0000","last_peered":"2024-05-08T21:53:23.703125+0000","last_clean":"2024-05-08T21:53:23.703125+0000","last_became_active":"2024-05-08T21:53:23.585073+0000","last_became_peered":"2024-05-08T21:53:23.585073+0000","last_unstale":"2024-05-08T21:53:23.703125+0000","last_undegraded":"2024-05-08T21:53:23.703125+0000","last_fullsized":"2024-05-08T21:53:23.703125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:53:23.119703+0000","last_clean_scrub_stamp":"2024-05-08T21:53:23.119703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:33.572410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68409,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":403,"num_read_kb":277,"num_write":232,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":860160,"data_stored":410274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.495}]},{"osd":1,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69899999999999995}]},{"osd":2,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.71499999999999997}]},{"osd":7,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.59899999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.624},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.74299999999999999}]}]},{"osd":10,"up_from":67,"seq":287762808855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32632,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712520,"statfs":{"total":95995035648,"available":95961620480,"internally_reserved":0,"allocated":1556480,"data_stored":1062365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.437},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55900000000000005}]},{"osd":3,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1,"5min":1,"15min":1},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.627},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]},{"osd":9,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.68100000000000005}]}]},{"osd":9,"up_from":62,"seq":266287972377,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":410252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34300000000000003}]},{"osd":1,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.502},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439}]},{"osd":3,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.38}]},{"osd":4,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.54200000000000004}]},{"osd":6,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57399999999999995}]},{"osd":8,"last update":"Wed May 8 21:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56499999999999995}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.60099999999999998}]}]},{"osd":8,"up_from":55,"seq":236223201309,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":410299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.59399999999999997,"15min":0.56799999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.0030000000000001,"5min":5.0030000000000001,"15min":5.0030000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60099999999999998}]},{"osd":2,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.432,"15min":0.41699999999999998},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.496},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47499999999999998,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92100000000000004,"5min":0.996,"15min":0.996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83399999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.68200000000000005,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8819999999999997,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.69899999999999995}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed May 8 21:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.748},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.875}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430240,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1478656,"data_stored":1003792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.442,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69699999999999995}]},{"osd":2,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84499999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67200000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.749},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82699999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.80500000000000005}]},{"osd":6,"last update":"Wed May 8 21:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68600000000000005}]},{"osd":8,"last update":"Wed May 8 21:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93700000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.88500000000000001}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.871}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593764,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":410314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.28899999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.31}]},{"osd":1,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.437},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.871,"15min":0.871},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed May 8 21:53:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80000000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 21:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80200000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.873,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.45800000000000002}]},{"osd":10,"last update":"Wed May 8 21:53:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":472227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.439},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.78500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed May 8 21:53:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.749,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed May 8 21:53:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed May 8 21:53:44 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.41699999999999998,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.309,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed May 8 21:53:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.33700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.83599999999999997}]},{"osd":6,"last update":"Wed May 8 21:53:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.752}]},{"osd":7,"last update":"Wed May 8 21:53:38 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.50800000000000001,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.255,"15min":0.255},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.503,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 21:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.433,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.65000000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed May 8 21:53:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":411260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.53800000000000003,"15min":0.48199999999999998},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.50600000000000001,"15min":0.47199999999999998},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.5}]},{"osd":2,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.83599999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed May 8 21:52:27 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.47799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.625}]},{"osd":6,"last update":"Wed May 8 21:53:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41999999999999998}]},{"osd":7,"last update":"Wed May 8 21:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73199999999999998}]},{"osd":8,"last update":"Wed May 8 21:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58599999999999997}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":10,"last update":"Wed May 8 21:53:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67300000000000004}]},{"osd":11,"last update":"Wed May 8 21:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66200000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":472399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48999999999999999,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69799999999999995}]},{"osd":3,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73899999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed May 8 21:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed May 8 21:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed May 8 21:53:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71999999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82799999999999996,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed May 8 21:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52900000000000003}]},{"osd":8,"last update":"Wed May 8 21:54:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.62},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60599999999999998}]},{"osd":9,"last update":"Wed May 8 21:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":3,"up_from":24,"seq":103079215151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":884736,"data_stored":409794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.34999999999999998},"min":{"1min":0.28000000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65100000000000002,"5min":1.028,"15min":1.028},"last":0.27700000000000002}]},{"osd":1,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.433,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51100000000000001}]},{"osd":4,"last update":"Wed May 8 21:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.873,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.873,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed May 8 21:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42199999999999999}]},{"osd":7,"last update":"Wed May 8 21:53:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80800000000000005,"5min":1.056,"15min":1.056},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.54100000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92100000000000004,"5min":1.329,"15min":1.329},"last":0.47999999999999998}]},{"osd":8,"last update":"Wed May 8 21:52:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.627},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.48599999999999999}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.438},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.62}]},{"osd":10,"last update":"Wed May 8 21:53:33 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":410175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed May 8 21:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.28000000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.68799999999999994,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.623,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed May 8 21:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71299999999999997}]},{"osd":3,"last update":"Wed May 8 21:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Wed May 8 21:53:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.747},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed May 8 21:53:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed May 8 21:53:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39900000000000002}]},{"osd":8,"last update":"Wed May 8 21:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 21:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822696,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32564,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712588,"statfs":{"total":95995035648,"available":95961690112,"internally_reserved":0,"allocated":1486848,"data_stored":1002449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.31,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77000000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed May 8 21:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.48599999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71799999999999997,"5min":1.367,"15min":1.367},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40799999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed May 8 21:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.32200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63200000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57999999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed May 8 21:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.441},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed May 8 21:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":1.399,"15min":1.399},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed May 8 21:53:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed May 8 21:53:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.57199999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 21:53:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.41399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.748}]},{"osd":9,"last update":"Wed May 8 21:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73799999999999999}]},{"osd":10,"last update":"Wed May 8 21:53:31 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.6499999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.617,"5min":3.617,"15min":3.617},"last":0.873}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:54:07.818 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T21:54:07.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T21:54:07.818 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T21:54:07.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T21:54:08.081 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:08.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:08 smithi064 ceph-mon[42500]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:54:08.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:08 smithi160 ceph-mon[46962]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:54:08.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:08 smithi186 ceph-mon[46619]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 -- 172.21.15.64:0/1943082170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da40ffed0 msgr2=0x7f9da4100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 --2- 172.21.15.64:0/1943082170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da40ffed0 0x7f9da4100350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009990 tx=0x7f9d98009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 -- 172.21.15.64:0/1943082170 shutdown_connections 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 --2- 172.21.15.64:0/1943082170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da4100890 0x7f9da410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 --2- 172.21.15.64:0/1943082170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9da4105f00 0x7f9da41062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 --2- 172.21.15.64:0/1943082170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da40ffed0 0x7f9da4100350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 -- 172.21.15.64:0/1943082170 >> 172.21.15.64:0/1943082170 conn(0x7f9da4075f80 msgr2=0x7f9da4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 -- 172.21.15.64:0/1943082170 shutdown_connections 2024-05-08T21:54:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.024+0000 7f9daa189700 1 -- 172.21.15.64:0/1943082170 wait complete. 2024-05-08T21:54:09.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.025+0000 7f9daa189700 1 Processor -- start 2024-05-08T21:54:09.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.025+0000 7f9daa189700 1 -- start start 2024-05-08T21:54:09.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9daa189700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9da40ffed0 0x7f9da41974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9daa189700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 0x7f9da41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9daa189700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da4105f00 0x7f9da419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9daa189700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da419c3b0 con 0x7f9da4100890 2024-05-08T21:54:09.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9daa189700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da419c520 con 0x7f9da4105f00 2024-05-08T21:54:09.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9daa189700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da419c6e0 con 0x7f9da40ffed0 2024-05-08T21:54:09.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 0x7f9da41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 0x7f9da41979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50464/0 (socket says 172.21.15.64:50464) 2024-05-08T21:54:09.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 -- 172.21.15.64:0/1150621343 learned_addr learned my addr 172.21.15.64:0/1150621343 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:09.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da37fe700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9da40ffed0 0x7f9da41974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da3fff700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da4105f00 0x7f9da419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 -- 172.21.15.64:0/1150621343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9da40ffed0 msgr2=0x7f9da41974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9da40ffed0 0x7f9da41974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 -- 172.21.15.64:0/1150621343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da4105f00 msgr2=0x7f9da419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da4105f00 0x7f9da419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 -- 172.21.15.64:0/1150621343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980095f0 con 0x7f9da4100890 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da2ffd700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 0x7f9da41979f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d98000c00 tx=0x7f9d9800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.026+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801d070 con 0x7f9da4100890 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.027+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9800fd30 con 0x7f9da4100890 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.027+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da419c960 con 0x7f9da4100890 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.027+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98017630 con 0x7f9da4100890 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.027+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da41a4220 con 0x7f9da4100890 2024-05-08T21:54:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.028+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da4108420 con 0x7f9da4100890 2024-05-08T21:54:09.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.029+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9d98004830 con 0x7f9da4100890 2024-05-08T21:54:09.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.029+0000 7f9da0ff9700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d8c0a11b0 0x7f9d8c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.029+0000 7f9da37fe700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d8c0a11b0 0x7f9d8c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.029+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d980cdf00 con 0x7f9da4100890 2024-05-08T21:54:09.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.030+0000 7f9da37fe700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d8c0a11b0 0x7f9d8c0a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009a10 tx=0x7f9d94009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:09.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.033+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d98097770 con 0x7f9da4100890 2024-05-08T21:54:09.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.284+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9da404ec60 con 0x7f9da4100890 2024-05-08T21:54:09.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.284+0000 7f9da0ff9700 1 -- 172.21.15.64:0/1150621343 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9d9809ad90 con 0x7f9da4100890 2024-05-08T21:54:09.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:09.286 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d8c0a11b0 msgr2=0x7f9d8c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d8c0a11b0 0x7f9d8c0a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009a10 tx=0x7f9d94009190 comp rx=0 tx=0).stop 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 msgr2=0x7f9da41979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 0x7f9da41979f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d98000c00 tx=0x7f9d9800f7d0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 shutdown_connections 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9d8c0a11b0 0x7f9d8c0a3670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da4105f00 0x7f9da419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.286+0000 7f9daa189700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9da40ffed0 0x7f9da41974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.287+0000 7f9daa189700 1 --2- 172.21.15.64:0/1150621343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da4100890 0x7f9da41979f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.287+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 >> 172.21.15.64:0/1150621343 conn(0x7f9da4075f80 msgr2=0x7f9da40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:09.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.287+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 shutdown_connections 2024-05-08T21:54:09.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.287+0000 7f9daa189700 1 -- 172.21.15.64:0/1150621343 wait complete. 2024-05-08T21:54:09.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:09 smithi064 ceph-mon[42500]: pgmap v243: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T21:54:09.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:09 smithi160 ceph-mon[46962]: pgmap v243: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T21:54:09.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:09 smithi186 ceph-mon[46619]: pgmap v243: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T21:54:09.651 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T21:54:09.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:54:09.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 -- 172.21.15.64:0/2964314422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 msgr2=0x7f73801092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 --2- 172.21.15.64:0/2964314422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f73801092a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f737800b0a0 tx=0x7f737800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 -- 172.21.15.64:0/2964314422 shutdown_connections 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 --2- 172.21.15.64:0/2964314422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7380100e70 0x7f7380101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 --2- 172.21.15.64:0/2964314422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f73801092a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 --2- 172.21.15.64:0/2964314422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7380101fc0 0x7f7380102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 -- 172.21.15.64:0/2964314422 >> 172.21.15.64:0/2964314422 conn(0x7f73800fc400 msgr2=0x7f73800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 -- 172.21.15.64:0/2964314422 shutdown_connections 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.781+0000 7f7386b0f700 1 -- 172.21.15.64:0/2964314422 wait complete. 2024-05-08T21:54:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 Processor -- start 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 -- start start 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7380100e70 0x7f7380196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7380101fc0 0x7f7380197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f738019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019e950 con 0x7f7380102980 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019ea90 con 0x7f7380100e70 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f7386b0f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019ec20 con 0x7f7380101fc0 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f738019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f738019e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50486/0 (socket says 172.21.15.64:50486) 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 -- 172.21.15.64:0/1264038647 learned_addr learned my addr 172.21.15.64:0/1264038647 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f737ffff700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7380101fc0 0x7f7380197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73848ab700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7380100e70 0x7f7380196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 -- 172.21.15.64:0/1264038647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7380101fc0 msgr2=0x7f7380197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7380101fc0 0x7f7380197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 -- 172.21.15.64:0/1264038647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7380100e70 msgr2=0x7f7380196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7380100e70 0x7f7380196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.782+0000 7f73850ac700 1 -- 172.21.15.64:0/1264038647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7378009d00 con 0x7f7380102980 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.783+0000 7f73850ac700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f738019e380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f737800d010 tx=0x7f7378004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.783+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800e040 con 0x7f7380102980 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.783+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737801db50 con 0x7f7380102980 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.783+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738019eea0 con 0x7f7380102980 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.783+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737801bb60 con 0x7f7380102980 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.783+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738019f2c0 con 0x7f7380102980 2024-05-08T21:54:09.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.784+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7380108210 con 0x7f7380102980 2024-05-08T21:54:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.785+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f737801bd00 con 0x7f7380102980 2024-05-08T21:54:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.786+0000 7f737dffb700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73700a11b0 0x7f73700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.786+0000 7f73848ab700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73700a11b0 0x7f73700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.786+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73780ce5b0 con 0x7f7380102980 2024-05-08T21:54:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.786+0000 7f73848ab700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73700a11b0 0x7f73700a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7374005fd0 tx=0x7f737400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:09.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.790+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7378097d20 con 0x7f7380102980 2024-05-08T21:54:09.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.997+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f738010c670 con 0x7f7380102980 2024-05-08T21:54:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:09.999+0000 7f737dffb700 1 -- 172.21.15.64:0/1264038647 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22119 (secure 0 0 0) 0x7f7378006070 con 0x7f7380102980 2024-05-08T21:54:10.000 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:10.001 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:24.557465+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73700a11b0 msgr2=0x7f73700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73700a11b0 0x7f73700a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7374005fd0 tx=0x7f737400a560 comp rx=0 tx=0).stop 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 msgr2=0x7f738019e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f738019e380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f737800d010 tx=0x7f7378004d00 comp rx=0 tx=0).stop 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 shutdown_connections 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f73700a11b0 0x7f73700a3670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7380100e70 0x7f7380196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7380102980 0x7f738019e380 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 --2- 172.21.15.64:0/1264038647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7380101fc0 0x7f7380197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 >> 172.21.15.64:0/1264038647 conn(0x7f73800fc400 msgr2=0x7f7380105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 shutdown_connections 2024-05-08T21:54:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.001+0000 7f7386b0f700 1 -- 172.21.15.64:0/1264038647 wait complete. 2024-05-08T21:54:10.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:53:21.578075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:53:22.933075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:54:10.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:54:10.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.144+0000 7f5620a69700 1 -- 172.21.15.64:0/2831953953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c0ffe60 msgr2=0x7f561c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.144+0000 7f5620a69700 1 --2- 172.21.15.64:0/2831953953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c0ffe60 0x7f561c100280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 -- 172.21.15.64:0/2831953953 shutdown_connections 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 --2- 172.21.15.64:0/2831953953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f561c101970 0x7f561c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 --2- 172.21.15.64:0/2831953953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c0ffe60 0x7f561c100280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 --2- 172.21.15.64:0/2831953953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c100fb0 0x7f561c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 -- 172.21.15.64:0/2831953953 >> 172.21.15.64:0/2831953953 conn(0x7f561c0fb3f0 msgr2=0x7f561c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 -- 172.21.15.64:0/2831953953 shutdown_connections 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 -- 172.21.15.64:0/2831953953 wait complete. 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 Processor -- start 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 -- start start 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c0ffe60 0x7f561c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.145+0000 7f5620a69700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f561c100fb0 0x7f561c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5620a69700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 0x7f561c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5620a69700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c19ea60 con 0x7f561c101970 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5620a69700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c19eba0 con 0x7f561c0ffe60 2024-05-08T21:54:10.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5620a69700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c19ed30 con 0x7f561c100fb0 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 0x7f561c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 0x7f561c19e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50492/0 (socket says 172.21.15.64:50492) 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 -- 172.21.15.64:0/1163957116 learned_addr learned my addr 172.21.15.64:0/1163957116 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5619d9b700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f561c100fb0 0x7f561c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561a59c700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c0ffe60 0x7f561c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 -- 172.21.15.64:0/1163957116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f561c100fb0 msgr2=0x7f561c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f561c100fb0 0x7f561c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 -- 172.21.15.64:0/1163957116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c0ffe60 msgr2=0x7f561c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c0ffe60 0x7f561c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 -- 172.21.15.64:0/1163957116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0095f0 con 0x7f561c101970 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561a59c700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c0ffe60 0x7f561c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f561ad9d700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 0x7f561c19e490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5614007ee0 tx=0x7f561400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400ddd0 con 0x7f561c101970 2024-05-08T21:54:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f561400f040 con 0x7f561c101970 2024-05-08T21:54:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c19efb0 con 0x7f561c101970 2024-05-08T21:54:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140048e0 con 0x7f561c101970 2024-05-08T21:54:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.146+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c19f490 con 0x7f561c101970 2024-05-08T21:54:10.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.147+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c067cf0 con 0x7f561c101970 2024-05-08T21:54:10.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.148+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5614003a00 con 0x7f561c101970 2024-05-08T21:54:10.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.149+0000 7f560b7fe700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56040a11b0 0x7f56040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.149+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56140cc230 con 0x7f561c101970 2024-05-08T21:54:10.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.149+0000 7f561a59c700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56040a11b0 0x7f56040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.149+0000 7f561a59c700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56040a11b0 0x7f56040a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f560c0037f0 tx=0x7f560c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.153+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5614095910 con 0x7f561c101970 2024-05-08T21:54:10.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.362+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f561c10b860 con 0x7f561c101970 2024-05-08T21:54:10.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.362+0000 7f560b7fe700 1 -- 172.21.15.64:0/1163957116 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5614098f30 con 0x7f561c101970 2024-05-08T21:54:10.364 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56040a11b0 msgr2=0x7f56040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56040a11b0 0x7f56040a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f560c0037f0 tx=0x7f560c005fb0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 msgr2=0x7f561c19e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 0x7f561c19e490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5614007ee0 tx=0x7f561400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 shutdown_connections 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f56040a11b0 0x7f56040a3670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f561c0ffe60 0x7f561c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f561c101970 0x7f561c19e490 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 --2- 172.21.15.64:0/1163957116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f561c100fb0 0x7f561c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 >> 172.21.15.64:0/1163957116 conn(0x7f561c0fb3f0 msgr2=0x7f561c067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 shutdown_connections 2024-05-08T21:54:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.365+0000 7f5620a69700 1 -- 172.21.15.64:0/1163957116 wait complete. 2024-05-08T21:54:10.378 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:10 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1150621343' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T21:54:10.378 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:10 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1264038647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:10.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:54:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 -- 172.21.15.64:0/2781857293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4100fb0 msgr2=0x7feea4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 --2- 172.21.15.64:0/2781857293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4100fb0 0x7feea4101430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fee90009990 tx=0x7fee90009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 -- 172.21.15.64:0/2781857293 shutdown_connections 2024-05-08T21:54:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 --2- 172.21.15.64:0/2781857293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea40ffe60 0x7feea4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 --2- 172.21.15.64:0/2781857293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feea4101970 0x7feea4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 --2- 172.21.15.64:0/2781857293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4100fb0 0x7feea4101430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 -- 172.21.15.64:0/2781857293 >> 172.21.15.64:0/2781857293 conn(0x7feea40fb3f0 msgr2=0x7feea40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 -- 172.21.15.64:0/2781857293 shutdown_connections 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.507+0000 7feeaaac9700 1 -- 172.21.15.64:0/2781857293 wait complete. 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 Processor -- start 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 -- start start 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feea40ffe60 0x7feea4068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea4100fb0 0x7feea419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 0x7feea419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea419ea20 con 0x7feea4101970 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea419eb90 con 0x7feea4100fb0 2024-05-08T21:54:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feeaaac9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea419ed50 con 0x7feea40ffe60 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea9066700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 0x7feea419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea9066700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 0x7feea419e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50508/0 (socket says 172.21.15.64:50508) 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea9066700 1 -- 172.21.15.64:0/3579119327 learned_addr learned my addr 172.21.15.64:0/3579119327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea8865700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feea40ffe60 0x7feea4068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea3fff700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea4100fb0 0x7feea419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea9066700 1 -- 172.21.15.64:0/3579119327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feea40ffe60 msgr2=0x7feea4068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.508+0000 7feea9066700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feea40ffe60 0x7feea4068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea9066700 1 -- 172.21.15.64:0/3579119327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea4100fb0 msgr2=0x7feea419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea9066700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea4100fb0 0x7feea419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea9066700 1 -- 172.21.15.64:0/3579119327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee900095f0 con 0x7feea4101970 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea3fff700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea4100fb0 0x7feea419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea9066700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 0x7feea419e230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00ada0 tx=0x7fee9c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f070 con 0x7feea4101970 2024-05-08T21:54:10.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee9c007b30 con 0x7feea4101970 2024-05-08T21:54:10.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea419efd0 con 0x7feea4101970 2024-05-08T21:54:10.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0053e0 con 0x7feea4101970 2024-05-08T21:54:10.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.509+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea419f400 con 0x7feea4101970 2024-05-08T21:54:10.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.510+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea4107320 con 0x7feea4101970 2024-05-08T21:54:10.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.511+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fee9c00c980 con 0x7feea4101970 2024-05-08T21:54:10.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.512+0000 7feea1ffb700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fee940a11b0 0x7fee940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.512+0000 7feea8865700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fee940a11b0 0x7fee940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.512+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee9c0cc520 con 0x7feea4101970 2024-05-08T21:54:10.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.513+0000 7feea8865700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fee940a11b0 0x7fee940a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fee98005950 tx=0x7fee9800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:10.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.516+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fee9c095d10 con 0x7feea4101970 2024-05-08T21:54:10.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:10 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1150621343' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T21:54:10.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:10 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1264038647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:10.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:10 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1150621343' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T21:54:10.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:10 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1264038647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:10.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.726+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feea410b660 con 0x7feea4101970 2024-05-08T21:54:10.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.727+0000 7feea1ffb700 1 -- 172.21.15.64:0/3579119327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee9c099330 con 0x7feea4101970 2024-05-08T21:54:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-08T21:54:10.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fee940a11b0 msgr2=0x7fee940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fee940a11b0 0x7fee940a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fee98005950 tx=0x7fee9800a300 comp rx=0 tx=0).stop 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 msgr2=0x7feea419e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 0x7feea419e230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00ada0 tx=0x7fee9c00c330 comp rx=0 tx=0).stop 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 shutdown_connections 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fee940a11b0 0x7fee940a3670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea4101970 0x7feea419e230 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feea40ffe60 0x7feea4068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 --2- 172.21.15.64:0/3579119327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea4100fb0 0x7feea419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 >> 172.21.15.64:0/3579119327 conn(0x7feea40fb3f0 msgr2=0x7feea4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 shutdown_connections 2024-05-08T21:54:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.729+0000 7feeaaac9700 1 -- 172.21.15.64:0/3579119327 wait complete. 2024-05-08T21:54:10.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:54:10.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 -- 172.21.15.64:0/845103906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478101980 msgr2=0x7f54781082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 --2- 172.21.15.64:0/845103906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478101980 0x7f54781082a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f547000b0a0 tx=0x7f547000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 -- 172.21.15.64:0/845103906 shutdown_connections 2024-05-08T21:54:10.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 --2- 172.21.15.64:0/845103906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478101980 0x7f54781082a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 --2- 172.21.15.64:0/845103906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54780ffe70 0x7f5478100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 --2- 172.21.15.64:0/845103906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5478100fc0 0x7f5478101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 -- 172.21.15.64:0/845103906 >> 172.21.15.64:0/845103906 conn(0x7f54780fb400 msgr2=0x7f54780fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:10.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 -- 172.21.15.64:0/845103906 shutdown_connections 2024-05-08T21:54:10.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.879+0000 7f547f4f5700 1 -- 172.21.15.64:0/845103906 wait complete. 2024-05-08T21:54:10.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 Processor -- start 2024-05-08T21:54:10.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 -- start start 2024-05-08T21:54:10.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54780ffe70 0x7f5478196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 0x7f5478197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478101980 0x7f547819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547819eb00 con 0x7f5478100fc0 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547819ec40 con 0x7f54780ffe70 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.880+0000 7f547f4f5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547819edd0 con 0x7f5478101980 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 0x7f5478197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 0x7f5478197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50534/0 (socket says 172.21.15.64:50534) 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 -- 172.21.15.64:0/1294796421 learned_addr learned my addr 172.21.15.64:0/1294796421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547d291700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54780ffe70 0x7f5478196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 -- 172.21.15.64:0/1294796421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478101980 msgr2=0x7f547819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478101980 0x7f547819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 -- 172.21.15.64:0/1294796421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54780ffe70 msgr2=0x7f5478196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54780ffe70 0x7f5478196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 -- 172.21.15.64:0/1294796421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009d00 con 0x7f5478100fc0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547ca90700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 0x7f5478197510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f547400c920 tx=0x7f547400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474007b00 con 0x7f5478100fc0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5474004cf0 con 0x7f5478100fc0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547819f0b0 con 0x7f5478100fc0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474005610 con 0x7f5478100fc0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.881+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547819f610 con 0x7f5478100fc0 2024-05-08T21:54:10.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.882+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5478107210 con 0x7f5478100fc0 2024-05-08T21:54:10.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.883+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5474004750 con 0x7f5478100fc0 2024-05-08T21:54:10.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.884+0000 7f546e7fc700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a1230 0x7f54640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:10.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.884+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54740cc900 con 0x7f5478100fc0 2024-05-08T21:54:10.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.884+0000 7f547d291700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a1230 0x7f54640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:10.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.884+0000 7f547d291700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a1230 0x7f54640a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5468005950 tx=0x7f54680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:10.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:10.888+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5474095fe0 con 0x7f5478100fc0 2024-05-08T21:54:11.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.093+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f547810c800 con 0x7f5478100fc0 2024-05-08T21:54:11.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.094+0000 7f546e7fc700 1 -- 172.21.15.64:0/1294796421 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f547401d020 con 0x7f5478100fc0 2024-05-08T21:54:11.095 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-08T21:54:11.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a1230 msgr2=0x7f54640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a1230 0x7f54640a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5468005950 tx=0x7f54680058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 msgr2=0x7f5478197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 0x7f5478197510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f547400c920 tx=0x7f547400cce0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 shutdown_connections 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f54640a1230 0x7f54640a36f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54780ffe70 0x7f5478196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5478101980 0x7f547819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 --2- 172.21.15.64:0/1294796421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5478100fc0 0x7f5478197510 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 >> 172.21.15.64:0/1294796421 conn(0x7f54780fb400 msgr2=0x7f5478104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 shutdown_connections 2024-05-08T21:54:11.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.096+0000 7f547f4f5700 1 -- 172.21.15.64:0/1294796421 wait complete. 2024-05-08T21:54:11.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 -- 172.21.15.64:0/3082842591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 msgr2=0x7f978c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 --2- 172.21.15.64:0/3082842591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c1081c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f978400b0d0 tx=0x7f978400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 -- 172.21.15.64:0/3082842591 shutdown_connections 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 --2- 172.21.15.64:0/3082842591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c1081c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 --2- 172.21.15.64:0/3082842591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0ffd80 0x7f978c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 --2- 172.21.15.64:0/3082842591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f978c100f70 0x7f978c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 -- 172.21.15.64:0/3082842591 >> 172.21.15.64:0/3082842591 conn(0x7f978c0fb2f0 msgr2=0x7f978c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 -- 172.21.15.64:0/3082842591 shutdown_connections 2024-05-08T21:54:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.236+0000 7f9791147700 1 -- 172.21.15.64:0/3082842591 wait complete. 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 Processor -- start 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 -- start start 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0ffd80 0x7f978c196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f978c100f70 0x7f978c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c19ea90 con 0x7f978c101910 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c19ebd0 con 0x7f978c100f70 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f9791147700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c19ed60 con 0x7f978c0ffd80 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f978b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.237+0000 7f978b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c19e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50556/0 (socket says 172.21.15.64:50556) 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 -- 172.21.15.64:0/2538029708 learned_addr learned my addr 172.21.15.64:0/2538029708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:11.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978ad9d700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0ffd80 0x7f978c196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978a59c700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f978c100f70 0x7f978c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 -- 172.21.15.64:0/2538029708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0ffd80 msgr2=0x7f978c196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0ffd80 0x7f978c196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 -- 172.21.15.64:0/2538029708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f978c100f70 msgr2=0x7f978c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f978c100f70 0x7f978c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 -- 172.21.15.64:0/2538029708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784009d00 con 0x7f978c101910 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f978b59e700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c19e4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9784007bd0 tx=0x7f9784009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978400e040 con 0x7f978c101910 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c19efe0 con 0x7f978c101910 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978401dd50 con 0x7f978c101910 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840123f0 con 0x7f978c101910 2024-05-08T21:54:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.238+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c19f400 con 0x7f978c101910 2024-05-08T21:54:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.239+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c067380 con 0x7f978c101910 2024-05-08T21:54:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.240+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f978401b550 con 0x7f978c101910 2024-05-08T21:54:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.241+0000 7f9773fff700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97740a11b0 0x7f97740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.241+0000 7f978ad9d700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97740a11b0 0x7f97740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.241+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97840cef80 con 0x7f978c101910 2024-05-08T21:54:11.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.241+0000 7f978ad9d700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97740a11b0 0x7f97740a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f978c100dd0 tx=0x7f977c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:11.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.244+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97840986f0 con 0x7f978c101910 2024-05-08T21:54:11.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:11 smithi064 ceph-mon[42500]: pgmap v244: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:11.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:11 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1163957116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:11 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3579119327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:11 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1294796421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.498+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f978c10b840 con 0x7f978c101910 2024-05-08T21:54:11.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.499+0000 7f9773fff700 1 -- 172.21.15.64:0/2538029708 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10921 (secure 0 0 0) 0x7f978409bd10 con 0x7f978c101910 2024-05-08T21:54:11.500 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:11.500 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T21:53:29.959964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:54:11.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97740a11b0 msgr2=0x7f97740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97740a11b0 0x7f97740a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f978c100dd0 tx=0x7f977c00b410 comp rx=0 tx=0).stop 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 msgr2=0x7f978c19e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c19e4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9784007bd0 tx=0x7f9784009440 comp rx=0 tx=0).stop 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 shutdown_connections 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97740a11b0 0x7f97740a3670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f978c101910 0x7f978c19e4c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0ffd80 0x7f978c196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 --2- 172.21.15.64:0/2538029708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f978c100f70 0x7f978c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 >> 172.21.15.64:0/2538029708 conn(0x7f978c0fb2f0 msgr2=0x7f978c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 shutdown_connections 2024-05-08T21:54:11.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.501+0000 7f9791147700 1 -- 172.21.15.64:0/2538029708 wait complete. 2024-05-08T21:54:11.504 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:54:11.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:54:11.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:11 smithi160 ceph-mon[46962]: pgmap v244: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:11.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:11 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1163957116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:11 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3579119327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:11 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1294796421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:11 smithi186 ceph-mon[46619]: pgmap v244: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:11 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1163957116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:11 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3579119327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:11 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1294796421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3423904577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c41018f0 msgr2=0x7fe7c41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3423904577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c41018f0 0x7fe7c41081a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00b0d0 tx=0x7fe7bc00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3423904577 shutdown_connections 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3423904577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c41018f0 0x7fe7c41081a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3423904577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3423904577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c4100f50 0x7fe7c41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3423904577 >> 172.21.15.64:0/3423904577 conn(0x7fe7c40fb2f0 msgr2=0x7fe7c40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3423904577 shutdown_connections 2024-05-08T21:54:11.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.643+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3423904577 wait complete. 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 Processor -- start 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 -- start start 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c4100f50 0x7fe7c419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c41018f0 0x7fe7c41a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41a0bb0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41a0d20 con 0x7fe7c41018f0 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cc69f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41a0ee0 con 0x7fe7c4100f50 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50580/0 (socket says 172.21.15.64:50580) 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 -- 172.21.15.64:0/3870448414 learned_addr learned my addr 172.21.15.64:0/3870448414 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7c9c3a700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c4100f50 0x7fe7c419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7cac3c700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c41018f0 0x7fe7c41a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 -- 172.21.15.64:0/3870448414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c4100f50 msgr2=0x7fe7c419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c4100f50 0x7fe7c419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 -- 172.21.15.64:0/3870448414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c41018f0 msgr2=0x7fe7c41a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c41018f0 0x7fe7c41a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 -- 172.21.15.64:0/3870448414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc009d00 con 0x7fe7c40ffd60 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7c9c3a700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c4100f50 0x7fe7c419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.644+0000 7fe7ca43b700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4068b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000d980 tx=0x7fe7c000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.645+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000de60 con 0x7fe7c40ffd60 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.645+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c41a11c0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.645+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7c00044e0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.645+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0003be0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.645+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c41a16d0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.646+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c4107110 con 0x7fe7c40ffd60 2024-05-08T21:54:11.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.647+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe7c0009bb0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.648+0000 7fe7b77fe700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe7b00a11b0 0x7fe7b00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:11.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.648+0000 7fe7c9c3a700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe7b00a11b0 0x7fe7b00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:11.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.648+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7c00cc9d0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.648+0000 7fe7c9c3a700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe7b00a11b0 0x7fe7b00a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005950 tx=0x7fe7b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:11.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.652+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe7c00961b0 con 0x7fe7c40ffd60 2024-05-08T21:54:11.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.862+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7c410b580 con 0x7fe7c40ffd60 2024-05-08T21:54:11.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.863+0000 7fe7b77fe700 1 -- 172.21.15.64:0/3870448414 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22119 (secure 0 0 0) 0x7fe7c0009940 con 0x7fe7c40ffd60 2024-05-08T21:54:11.866 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:11.866 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:24.557465+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:54:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe7b00a11b0 msgr2=0x7fe7b00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe7b00a11b0 0x7fe7b00a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005950 tx=0x7fe7b80058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 msgr2=0x7fe7c4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4068b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000d980 tx=0x7fe7c000dd40 comp rx=0 tx=0).stop 2024-05-08T21:54:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 shutdown_connections 2024-05-08T21:54:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe7b00a11b0 0x7fe7b00a3670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c41018f0 0x7fe7c41a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7c40ffd60 0x7fe7c4068b10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 --2- 172.21.15.64:0/3870448414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c4100f50 0x7fe7c419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 >> 172.21.15.64:0/3870448414 conn(0x7fe7c40fb2f0 msgr2=0x7fe7c4104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 shutdown_connections 2024-05-08T21:54:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:11.866+0000 7fe7cc69f700 1 -- 172.21.15.64:0/3870448414 wait complete. 2024-05-08T21:54:11.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:53:21.578075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:53:22.933075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:54:11.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:54:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 -- 172.21.15.64:0/2601156692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24101a60 msgr2=0x7f6a24103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 --2- 172.21.15.64:0/2601156692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24101a60 0x7f6a24103e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009990 tx=0x7f6a18009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 -- 172.21.15.64:0/2601156692 shutdown_connections 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 --2- 172.21.15.64:0/2601156692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a24106cc0 0x7f6a241090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 --2- 172.21.15.64:0/2601156692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24101a60 0x7f6a24103e50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 --2- 172.21.15.64:0/2601156692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24104390 0x7f6a24106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 -- 172.21.15.64:0/2601156692 >> 172.21.15.64:0/2601156692 conn(0x7f6a240fb3f0 msgr2=0x7f6a240fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 -- 172.21.15.64:0/2601156692 shutdown_connections 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.010+0000 7f6a2998e700 1 -- 172.21.15.64:0/2601156692 wait complete. 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 Processor -- start 2024-05-08T21:54:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 -- start start 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a24101a60 0x7f6a24196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 0x7f6a241973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24106cc0 0x7f6a2419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2419e9c0 con 0x7f6a24104390 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2419eb00 con 0x7f6a24101a60 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a2998e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2419ec90 con 0x7f6a24106cc0 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a227fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 0x7f6a241973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a227fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 0x7f6a241973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50590/0 (socket says 172.21.15.64:50590) 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a227fc700 1 -- 172.21.15.64:0/3875767490 learned_addr learned my addr 172.21.15.64:0/3875767490 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a22ffd700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a24101a60 0x7f6a24196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.011+0000 7f6a237fe700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24106cc0 0x7f6a2419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a227fc700 1 -- 172.21.15.64:0/3875767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24106cc0 msgr2=0x7f6a2419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a227fc700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24106cc0 0x7f6a2419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a227fc700 1 -- 172.21.15.64:0/3875767490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a24101a60 msgr2=0x7f6a24196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a227fc700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a24101a60 0x7f6a24196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a227fc700 1 -- 172.21.15.64:0/3875767490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a180095f0 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a237fe700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24106cc0 0x7f6a2419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a227fc700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 0x7f6a241973d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400d980 tx=0x7f6a1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004220 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2419ef70 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a14007640 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14005610 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.012+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a240ff850 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.013+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a240fcf70 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.014+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6a14004750 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.015+0000 7f6a2898c700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6a0c0a55c0 0x7f6a0c0a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.015+0000 7f6a22ffd700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6a0c0a55c0 0x7f6a0c0a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.015+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a140cc8b0 con 0x7f6a24104390 2024-05-08T21:54:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.016+0000 7f6a22ffd700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6a0c0a55c0 0x7f6a0c0a7a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b5c0 tx=0x7f6a18003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:12.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.020+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6a14095f90 con 0x7f6a24104390 2024-05-08T21:54:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.225+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a24067cc0 con 0x7f6a24104390 2024-05-08T21:54:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.225+0000 7f6a2898c700 1 -- 172.21.15.64:0/3875767490 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a14016020 con 0x7f6a24104390 2024-05-08T21:54:12.227 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6a0c0a55c0 msgr2=0x7f6a0c0a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6a0c0a55c0 0x7f6a0c0a7a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b5c0 tx=0x7f6a18003790 comp rx=0 tx=0).stop 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 msgr2=0x7f6a241973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 0x7f6a241973d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400d980 tx=0x7f6a1400dc90 comp rx=0 tx=0).stop 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 shutdown_connections 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6a0c0a55c0 0x7f6a0c0a7a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a24101a60 0x7f6a24196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24106cc0 0x7f6a2419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 --2- 172.21.15.64:0/3875767490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24104390 0x7f6a241973d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 >> 172.21.15.64:0/3875767490 conn(0x7f6a240fb3f0 msgr2=0x7f6a24102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 shutdown_connections 2024-05-08T21:54:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.227+0000 7f6a2998e700 1 -- 172.21.15.64:0/3875767490 wait complete. 2024-05-08T21:54:12.241 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 -- 172.21.15.64:0/4129568668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9881019e0 msgr2=0x7fc988103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 --2- 172.21.15.64:0/4129568668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9881019e0 0x7fc988103dd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc978009990 tx=0x7fc978009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 -- 172.21.15.64:0/4129568668 shutdown_connections 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 --2- 172.21.15.64:0/4129568668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc988106c40 0x7fc9881090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 --2- 172.21.15.64:0/4129568668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9881019e0 0x7fc988103dd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 --2- 172.21.15.64:0/4129568668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc988104310 0x7fc988106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 -- 172.21.15.64:0/4129568668 >> 172.21.15.64:0/4129568668 conn(0x7fc9880fb3f0 msgr2=0x7fc9880fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 -- 172.21.15.64:0/4129568668 shutdown_connections 2024-05-08T21:54:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.370+0000 7fc98cb59700 1 -- 172.21.15.64:0/4129568668 wait complete. 2024-05-08T21:54:12.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 Processor -- start 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 -- start start 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9881019e0 0x7fc988196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc988104310 0x7fc988197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 0x7fc98819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98819ea60 con 0x7fc988106c40 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98819eba0 con 0x7fc988104310 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98cb59700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98819ed30 con 0x7fc9881019e0 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc986d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 0x7fc98819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc986d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 0x7fc98819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36560/0 (socket says 172.21.15.64:36560) 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc986d9d700 1 -- 172.21.15.64:0/3633700212 learned_addr learned my addr 172.21.15.64:0/3633700212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc985d9b700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc988104310 0x7fc988197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc98659c700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9881019e0 0x7fc988196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.371+0000 7fc986d9d700 1 -- 172.21.15.64:0/3633700212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9881019e0 msgr2=0x7fc988196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc986d9d700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9881019e0 0x7fc988196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc986d9d700 1 -- 172.21.15.64:0/3633700212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc988104310 msgr2=0x7fc988197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc986d9d700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc988104310 0x7fc988197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc986d9d700 1 -- 172.21.15.64:0/3633700212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9780095f0 con 0x7fc988106c40 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc986d9d700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 0x7fc98819e490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc980007ee0 tx=0x7fc98000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98000dd20 con 0x7fc988106c40 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc98000f040 con 0x7fc988106c40 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98819efb0 con 0x7fc988106c40 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800048a0 con 0x7fc988106c40 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.372+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98819f440 con 0x7fc988106c40 2024-05-08T21:54:12.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.373+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9880fcf70 con 0x7fc988106c40 2024-05-08T21:54:12.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.374+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc980003a00 con 0x7fc988106c40 2024-05-08T21:54:12.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.375+0000 7fc9777fe700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9700a11b0 0x7fc9700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.375+0000 7fc98659c700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9700a11b0 0x7fc9700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.375+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9800cc0b0 con 0x7fc988106c40 2024-05-08T21:54:12.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.375+0000 7fc98659c700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9700a11b0 0x7fc9700a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b5c0 tx=0x7fc978005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:12.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.379+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc980095890 con 0x7fc988106c40 2024-05-08T21:54:12.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:12 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2538029708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:12.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:12 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3870448414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.588+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc98810c790 con 0x7fc988106c40 2024-05-08T21:54:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.588+0000 7fc9777fe700 1 -- 172.21.15.64:0/3633700212 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc980098eb0 con 0x7fc988106c40 2024-05-08T21:54:12.590 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-08T21:54:12.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9700a11b0 msgr2=0x7fc9700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9700a11b0 0x7fc9700a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b5c0 tx=0x7fc978005fd0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 msgr2=0x7fc98819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 0x7fc98819e490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc980007ee0 tx=0x7fc98000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 shutdown_connections 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9700a11b0 0x7fc9700a3670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc988106c40 0x7fc98819e490 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9881019e0 0x7fc988196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 --2- 172.21.15.64:0/3633700212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc988104310 0x7fc988197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 >> 172.21.15.64:0/3633700212 conn(0x7fc9880fb3f0 msgr2=0x7fc9881024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 shutdown_connections 2024-05-08T21:54:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.591+0000 7fc98cb59700 1 -- 172.21.15.64:0/3633700212 wait complete. 2024-05-08T21:54:12.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:12 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2538029708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:12.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:12 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3870448414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:12.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:54:12.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:12 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2538029708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:12.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:12 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3870448414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 -- 172.21.15.64:0/3565042527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 msgr2=0x7fca08100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3565042527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08100280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009990 tx=0x7fc9f8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 -- 172.21.15.64:0/3565042527 shutdown_connections 2024-05-08T21:54:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3565042527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca08101970 0x7fca08108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3565042527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08100280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3565042527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08100fb0 0x7fca08101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 -- 172.21.15.64:0/3565042527 >> 172.21.15.64:0/3565042527 conn(0x7fca080fb3f0 msgr2=0x7fca080fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 -- 172.21.15.64:0/3565042527 shutdown_connections 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.733+0000 7fca0cc51700 1 -- 172.21.15.64:0/3565042527 wait complete. 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 Processor -- start 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 -- start start 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca08100fb0 0x7fca08196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08101970 0x7fca0819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0819d8f0 con 0x7fca080ffe60 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0819da60 con 0x7fca08101970 2024-05-08T21:54:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0cc51700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0819dc20 con 0x7fca08100fb0 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36578/0 (socket says 172.21.15.64:36578) 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 -- 172.21.15.64:0/3595651999 learned_addr learned my addr 172.21.15.64:0/3595651999 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca06d9d700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08101970 0x7fca0819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 -- 172.21.15.64:0/3595651999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca08100fb0 msgr2=0x7fca08196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca08100fb0 0x7fca08196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 -- 172.21.15.64:0/3595651999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08101970 msgr2=0x7fca0819d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08101970 0x7fca0819d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca0659c700 1 -- 172.21.15.64:0/3595651999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80095f0 con 0x7fca080ffe60 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.734+0000 7fca06d9d700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08101970 0x7fca0819d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.735+0000 7fca0659c700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08195e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b5c0 tx=0x7fc9f8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:12.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.735+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f801d070 con 0x7fca080ffe60 2024-05-08T21:54:12.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.735+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9f8004060 con 0x7fca080ffe60 2024-05-08T21:54:12.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.735+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0819dea0 con 0x7fca080ffe60 2024-05-08T21:54:12.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.735+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8022e10 con 0x7fca080ffe60 2024-05-08T21:54:12.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.735+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0819e260 con 0x7fca080ffe60 2024-05-08T21:54:12.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.736+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca081072d0 con 0x7fca080ffe60 2024-05-08T21:54:12.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.737+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc9f8022570 con 0x7fca080ffe60 2024-05-08T21:54:12.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.737+0000 7fc9ff7fe700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9f40a11b0 0x7fc9f40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:12.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.738+0000 7fca05d9b700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9f40a11b0 0x7fc9f40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:12.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.738+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9f80cdda0 con 0x7fca080ffe60 2024-05-08T21:54:12.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.738+0000 7fca05d9b700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9f40a11b0 0x7fc9f40a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005950 tx=0x7fc9f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:12.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.742+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc9f80975a0 con 0x7fca080ffe60 2024-05-08T21:54:12.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.947+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca0810b7e0 con 0x7fca080ffe60 2024-05-08T21:54:12.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.948+0000 7fc9ff7fe700 1 -- 172.21.15.64:0/3595651999 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9f809abc0 con 0x7fca080ffe60 2024-05-08T21:54:12.949 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.949+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9f40a11b0 msgr2=0x7fc9f40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.949+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9f40a11b0 0x7fc9f40a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005950 tx=0x7fc9f00058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 msgr2=0x7fca08195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08195e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b5c0 tx=0x7fc9f8005e80 comp rx=0 tx=0).stop 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 shutdown_connections 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9f40a11b0 0x7fc9f40a3670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca08101970 0x7fca0819d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca080ffe60 0x7fca08195e00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 --2- 172.21.15.64:0/3595651999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca08100fb0 0x7fca08196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 >> 172.21.15.64:0/3595651999 conn(0x7fca080fb3f0 msgr2=0x7fca08104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 shutdown_connections 2024-05-08T21:54:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:12.950+0000 7fca0cc51700 1 -- 172.21.15.64:0/3595651999 wait complete. 2024-05-08T21:54:12.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:54:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 -- 172.21.15.64:0/646945848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c0ffd60 msgr2=0x7f839c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 --2- 172.21.15.64:0/646945848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c0ffd60 0x7f839c100180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f838c0099c0 tx=0x7f838c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 -- 172.21.15.64:0/646945848 shutdown_connections 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 --2- 172.21.15.64:0/646945848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c1018f0 0x7f839c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 --2- 172.21.15.64:0/646945848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c0ffd60 0x7f839c100180 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 --2- 172.21.15.64:0/646945848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c100f50 0x7f839c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 -- 172.21.15.64:0/646945848 >> 172.21.15.64:0/646945848 conn(0x7f839c0fb2f0 msgr2=0x7f839c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 -- 172.21.15.64:0/646945848 shutdown_connections 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.091+0000 7f83a35b2700 1 -- 172.21.15.64:0/646945848 wait complete. 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 Processor -- start 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 -- start start 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c0ffd60 0x7f839c196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c100f50 0x7f839c197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 0x7f839c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c19ea30 con 0x7f839c1018f0 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c19eb70 con 0x7f839c100f50 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a35b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c19ed00 con 0x7f839c0ffd60 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a1b4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 0x7f839c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a1b4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 0x7f839c19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36590/0 (socket says 172.21.15.64:36590) 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a1b4f700 1 -- 172.21.15.64:0/3503709215 learned_addr learned my addr 172.21.15.64:0/3503709215 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a0b4d700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c100f50 0x7f839c197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.092+0000 7f83a134e700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c0ffd60 0x7f839c196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a1b4f700 1 -- 172.21.15.64:0/3503709215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c0ffd60 msgr2=0x7f839c196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a1b4f700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c0ffd60 0x7f839c196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a1b4f700 1 -- 172.21.15.64:0/3503709215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c100f50 msgr2=0x7f839c197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a1b4f700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c100f50 0x7f839c197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a1b4f700 1 -- 172.21.15.64:0/3503709215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0095f0 con 0x7f839c1018f0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a134e700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c0ffd60 0x7f839c196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a0b4d700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c100f50 0x7f839c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a1b4f700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 0x7f839c19e460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f839400ada0 tx=0x7f839400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839400f070 con 0x7f839c1018f0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8394007b30 con 0x7f839c1018f0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c19efe0 con 0x7f839c1018f0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394005440 con 0x7f839c1018f0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.093+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c19f460 con 0x7f839c1018f0 2024-05-08T21:54:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.094+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c067c10 con 0x7f839c1018f0 2024-05-08T21:54:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.095+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8394004ad0 con 0x7f839c1018f0 2024-05-08T21:54:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.096+0000 7f83927fc700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f83880a1230 0x7f83880a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.096+0000 7f83a134e700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f83880a1230 0x7f83880a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.096+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83940cc480 con 0x7f839c1018f0 2024-05-08T21:54:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.096+0000 7f83a134e700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f83880a1230 0x7f83880a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f838c00b580 tx=0x7f838c004500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.099+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8394095cf0 con 0x7f839c1018f0 2024-05-08T21:54:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.356+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f839c10b890 con 0x7f839c1018f0 2024-05-08T21:54:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.357+0000 7f83927fc700 1 -- 172.21.15.64:0/3503709215 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10921 (secure 0 0 0) 0x7f839401a310 con 0x7f839c1018f0 2024-05-08T21:54:13.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:13.359 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T21:53:29.959964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:54:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.359+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f83880a1230 msgr2=0x7f83880a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.359+0000 7f83a35b2700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f83880a1230 0x7f83880a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f838c00b580 tx=0x7f838c004500 comp rx=0 tx=0).stop 2024-05-08T21:54:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.359+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 msgr2=0x7f839c19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.359+0000 7f83a35b2700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 0x7f839c19e460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f839400ada0 tx=0x7f839400c330 comp rx=0 tx=0).stop 2024-05-08T21:54:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 shutdown_connections 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f83880a1230 0x7f83880a36f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839c1018f0 0x7f839c19e460 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839c0ffd60 0x7f839c196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 --2- 172.21.15.64:0/3503709215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f839c100f50 0x7f839c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 >> 172.21.15.64:0/3503709215 conn(0x7f839c0fb2f0 msgr2=0x7f839c068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 shutdown_connections 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.360+0000 7f83a35b2700 1 -- 172.21.15.64:0/3503709215 wait complete. 2024-05-08T21:54:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:54:13.374 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 -- 172.21.15.64:0/2018652856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f00ffd10 msgr2=0x7fb0f0100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/2018652856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0100130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40099c0 tx=0x7fb0e4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 -- 172.21.15.64:0/2018652856 shutdown_connections 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/2018652856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f01018a0 0x7fb0f0108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/2018652856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0100130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/2018652856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f0100f00 0x7fb0f0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 -- 172.21.15.64:0/2018652856 >> 172.21.15.64:0/2018652856 conn(0x7fb0f00fb260 msgr2=0x7fb0f00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 -- 172.21.15.64:0/2018652856 shutdown_connections 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.499+0000 7fb0f6e71700 1 -- 172.21.15.64:0/2018652856 wait complete. 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 Processor -- start 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 -- start start 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 0x7fb0f0197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f01018a0 0x7fb0f019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019e980 con 0x7fb0f0100f00 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019eac0 con 0x7fb0f01018a0 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f6e71700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019ec50 con 0x7fb0f00ffd10 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 0x7fb0f0197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 0x7fb0f0197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36600/0 (socket says 172.21.15.64:36600) 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 -- 172.21.15.64:0/695192384 learned_addr learned my addr 172.21.15.64:0/695192384 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f4c0d700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f540e700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f01018a0 0x7fb0f019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 -- 172.21.15.64:0/695192384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f00ffd10 msgr2=0x7fb0f0196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 -- 172.21.15.64:0/695192384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f01018a0 msgr2=0x7fb0f019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f01018a0 0x7fb0f019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 -- 172.21.15.64:0/695192384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40095f0 con 0x7fb0f0100f00 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f4c0d700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0f540e700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f01018a0 0x7fb0f019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.500+0000 7fb0effff700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 0x7fb0f0197390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00d950 tx=0x7fb0dc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.501+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc009b40 con 0x7fb0f0100f00 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.501+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0dc009ca0 con 0x7fb0f0100f00 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.501+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f019ef30 con 0x7fb0f0100f00 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.501+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00f620 con 0x7fb0f0100f00 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.501+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f019f400 con 0x7fb0f0100f00 2024-05-08T21:54:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.502+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f0067ba0 con 0x7fb0f0100f00 2024-05-08T21:54:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.503+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb0dc010460 con 0x7fb0f0100f00 2024-05-08T21:54:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.503+0000 7fb0edffb700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb0e00a1230 0x7fb0e00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.503+0000 7fb0f4c0d700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb0e00a1230 0x7fb0e00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.503+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0dc0ccbd0 con 0x7fb0f0100f00 2024-05-08T21:54:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.504+0000 7fb0f4c0d700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb0e00a1230 0x7fb0e00a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b580 tx=0x7fb0e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.508+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb0dc0963b0 con 0x7fb0f0100f00 2024-05-08T21:54:13.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:13 smithi160 ceph-mon[46962]: pgmap v245: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:13.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:13 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3875767490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:13 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3633700212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:13 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3595651999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:13 smithi186 ceph-mon[46619]: pgmap v245: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:13 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3875767490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:13 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3633700212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:13 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3595651999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:13 smithi064 ceph-mon[42500]: pgmap v245: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T21:54:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:13 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3875767490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:13 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3633700212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:13 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3595651999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.716+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0f010b570 con 0x7fb0f0100f00 2024-05-08T21:54:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.718+0000 7fb0edffb700 1 -- 172.21.15.64:0/695192384 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22119 (secure 0 0 0) 0x7fb0dc020090 con 0x7fb0f0100f00 2024-05-08T21:54:13.720 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:13.720 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:53:24.557465+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb0e00a1230 msgr2=0x7fb0e00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb0e00a1230 0x7fb0e00a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b580 tx=0x7fb0e4005fb0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 msgr2=0x7fb0f0197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 0x7fb0f0197390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00d950 tx=0x7fb0dc00dd10 comp rx=0 tx=0).stop 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 shutdown_connections 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb0e00a1230 0x7fb0e00a36f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0f01018a0 0x7fb0f019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f00ffd10 0x7fb0f0196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 --2- 172.21.15.64:0/695192384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0100f00 0x7fb0f0197390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.720+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 >> 172.21.15.64:0/695192384 conn(0x7fb0f00fb260 msgr2=0x7fb0f0068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.721+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 shutdown_connections 2024-05-08T21:54:13.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.721+0000 7fb0f6e71700 1 -- 172.21.15.64:0/695192384 wait complete. 2024-05-08T21:54:13.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:54:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 -- 172.21.15.64:0/1481881824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 msgr2=0x7fef48108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1481881824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef48108fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b0a0 tx=0x7fef4000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 -- 172.21.15.64:0/1481881824 shutdown_connections 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1481881824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef48101910 0x7fef48103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1481881824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef48108fe0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1481881824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef48104240 0x7fef48106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 -- 172.21.15.64:0/1481881824 >> 172.21.15.64:0/1481881824 conn(0x7fef480fb320 msgr2=0x7fef480fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 -- 172.21.15.64:0/1481881824 shutdown_connections 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 -- 172.21.15.64:0/1481881824 wait complete. 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.861+0000 7fef4ff81700 1 Processor -- start 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 -- start start 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef48101910 0x7fef48196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef48104240 0x7fef481973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef4819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819e990 con 0x7fef48106b70 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819ead0 con 0x7fef48104240 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819ec60 con 0x7fef48101910 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef4819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef4819e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36618/0 (socket says 172.21.15.64:36618) 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 -- 172.21.15.64:0/1218209243 learned_addr learned my addr 172.21.15.64:0/1218209243 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4dd1d700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef48101910 0x7fef48196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4d51c700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef48104240 0x7fef481973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 -- 172.21.15.64:0/1218209243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef48101910 msgr2=0x7fef48196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef48101910 0x7fef48196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 -- 172.21.15.64:0/1218209243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef48104240 msgr2=0x7fef481973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef48104240 0x7fef481973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 -- 172.21.15.64:0/1218209243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40009d00 con 0x7fef48106b70 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4e51e700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef4819e3c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef400078c0 tx=0x7fef400078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000e040 con 0x7fef48106b70 2024-05-08T21:54:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef40004c50 con 0x7fef48106b70 2024-05-08T21:54:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.862+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4819eee0 con 0x7fef48106b70 2024-05-08T21:54:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.863+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400052a0 con 0x7fef48106b70 2024-05-08T21:54:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.863+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4819f300 con 0x7fef48106b70 2024-05-08T21:54:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.863+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef480fcea0 con 0x7fef48106b70 2024-05-08T21:54:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.865+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fef4001d470 con 0x7fef48106b70 2024-05-08T21:54:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.865+0000 7fef3effd700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fef340a11b0 0x7fef340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.865+0000 7fef4dd1d700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fef340a11b0 0x7fef340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.865+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fef400ce290 con 0x7fef48106b70 2024-05-08T21:54:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.866+0000 7fef4dd1d700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fef340a11b0 0x7fef340a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fef44009c60 tx=0x7fef44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:13.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:13.869+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fef40097b00 con 0x7fef48106b70 2024-05-08T21:54:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.122+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef48067380 con 0x7fef48106b70 2024-05-08T21:54:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.122+0000 7fef3effd700 1 -- 172.21.15.64:0/1218209243 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10921 (secure 0 0 0) 0x7fef4009b120 con 0x7fef48106b70 2024-05-08T21:54:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:14.124 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T21:53:29.959964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:54:14.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fef340a11b0 msgr2=0x7fef340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fef340a11b0 0x7fef340a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fef44009c60 tx=0x7fef44009260 comp rx=0 tx=0).stop 2024-05-08T21:54:14.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 msgr2=0x7fef4819e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef4819e3c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef400078c0 tx=0x7fef400078f0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 shutdown_connections 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fef340a11b0 0x7fef340a3670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef48106b70 0x7fef4819e3c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef48101910 0x7fef48196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 --2- 172.21.15.64:0/1218209243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef48104240 0x7fef481973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 >> 172.21.15.64:0/1218209243 conn(0x7fef480fb320 msgr2=0x7fef48102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 shutdown_connections 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.124+0000 7fef4ff81700 1 -- 172.21.15.64:0/1218209243 wait complete. 2024-05-08T21:54:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:54:14.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T21:53:23.555551+0000', 'modified': '2024-05-08T21:53:29.959964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24506, 'mds_1': 24484, 'mds_2': 24476, 'mds_3': 24427, 'mds_4': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6839/3760541428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 3760541428}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 3760541428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6841/1669365277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6840', 'nonce': 1669365277}, {'type': 'v1', 'addr': '172.21.15.64:6841', 'nonce': 1669365277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.160:6833/3542328594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6832', 'nonce': 3542328594}, {'type': 'v1', 'addr': '172.21.15.160:6833', 'nonce': 3542328594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.186:6833/1716244539', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6832', 'nonce': 1716244539}, {'type': 'v1', 'addr': '172.21.15.186:6833', 'nonce': 1716244539}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/770617410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 770617410}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 770617410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/4069999836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 4069999836}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 4069999836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/3888486173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 3888486173}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 3888486173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.160:6837/4233185014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 4233185014}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 4233185014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.186:6839/539285459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 539285459}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 539285459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/2466892222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 2466892222}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 2466892222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T21:54:14.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T21:54:14.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-05-08T21:54:14.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 -- 172.21.15.64:0/461576834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 msgr2=0x7f9324109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 --2- 172.21.15.64:0/461576834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f9324109500 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f931c00b0a0 tx=0x7f931c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 -- 172.21.15.64:0/461576834 shutdown_connections 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 --2- 172.21.15.64:0/461576834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f9324109500 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 --2- 172.21.15.64:0/461576834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93240ff240 0x7f93240ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 --2- 172.21.15.64:0/461576834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 0x7f9324102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 -- 172.21.15.64:0/461576834 >> 172.21.15.64:0/461576834 conn(0x7f93240fab50 msgr2=0x7f93240fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 -- 172.21.15.64:0/461576834 shutdown_connections 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.266+0000 7f932b20e700 1 -- 172.21.15.64:0/461576834 wait complete. 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 Processor -- start 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 -- start start 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93240ff240 0x7f9324196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 0x7f9324197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f932419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932419eaf0 con 0x7f9324102c50 2024-05-08T21:54:14.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932419ec30 con 0x7f93241022b0 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f932b20e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932419edc0 con 0x7f93240ff240 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f93297ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f932419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f93297ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f932419e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36642/0 (socket says 172.21.15.64:36642) 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f93297ab700 1 -- 172.21.15.64:0/3852282575 learned_addr learned my addr 172.21.15.64:0/3852282575 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f9328faa700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93240ff240 0x7f9324196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.267+0000 7f9323fff700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 0x7f9324197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f93297ab700 1 -- 172.21.15.64:0/3852282575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93240ff240 msgr2=0x7f9324196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f93297ab700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93240ff240 0x7f9324196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f93297ab700 1 -- 172.21.15.64:0/3852282575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 msgr2=0x7f9324197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f93297ab700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 0x7f9324197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f93297ab700 1 -- 172.21.15.64:0/3852282575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c009d00 con 0x7f9324102c50 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f9323fff700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 0x7f9324197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f93297ab700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f932419e520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f931c004d00 tx=0x7f931c004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f9321ffb700 1 -- 172.21.15.64:0/3852282575 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00e040 con 0x7f9324102c50 2024-05-08T21:54:14.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f9321ffb700 1 -- 172.21.15.64:0/3852282575 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931c01db50 con 0x7f9324102c50 2024-05-08T21:54:14.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932419f040 con 0x7f9324102c50 2024-05-08T21:54:14.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f9321ffb700 1 -- 172.21.15.64:0/3852282575 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c01bbe0 con 0x7f9324102c50 2024-05-08T21:54:14.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.268+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932419f510 con 0x7f9324102c50 2024-05-08T21:54:14.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.270+0000 7f9321ffb700 1 -- 172.21.15.64:0/3852282575 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f931c01bd40 con 0x7f9324102c50 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.271+0000 7f9321ffb700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f93140a1230 0x7f93140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.271+0000 7f9321ffb700 1 -- 172.21.15.64:0/3852282575 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f931c0ce740 con 0x7f9324102c50 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.271+0000 7f9328faa700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f93140a1230 0x7f93140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.271+0000 7f9328faa700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f93140a1230 0x7f93140a36f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9318005950 tx=0x7f93180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 Processor -- start 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 -- start start 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c10d100 con 0x55790c1b64f0 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c10aac0 con 0x55790c1b7810 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93373c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c1063b0 con 0x55790c1b6e50 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36656/0 (socket says 172.21.15.64:36656) 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 -- 172.21.15.64:0/3794532975 learned_addr learned my addr 172.21.15.64:0/3794532975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f93297ab700 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 -- 172.21.15.64:0/3794532975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 msgr2=0x55790c1b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9323fff700 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 -- 172.21.15.64:0/3794532975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 msgr2=0x55790c1b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.272+0000 7f9328faa700 1 -- 172.21.15.64:0/3794532975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55790bf87360 con 0x55790c1b64f0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f93297ab700 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f9328faa700 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b6910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f931800b570 tx=0x7f9318012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410072a604cd5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f930effd700 1 -- 172.21.15.64:0/3794532975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318011b50 con 0x55790c1b64f0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f930effd700 1 -- 172.21.15.64:0/3794532975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9318011cb0 con 0x55790c1b64f0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f930effd700 1 -- 172.21.15.64:0/3794532975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931801d610 con 0x55790c1b64f0 2024-05-08T21:54:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f93373c8b80 1 -- 172.21.15.64:0/3794532975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 msgr2=0x55790c1b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.273+0000 7f93373c8b80 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b6910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f931800b570 tx=0x7f9318012000 comp rx=0 tx=0).stop 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- 172.21.15.64:0/3794532975 shutdown_connections 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b6910 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 --2- 172.21.15.64:0/3794532975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- 172.21.15.64:0/3794532975 >> 172.21.15.64:0/3794532975 conn(0x55790c1141c0 msgr2=0x55790c1b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- 172.21.15.64:0/3794532975 shutdown_connections 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- 172.21.15.64:0/3794532975 wait complete. 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- start start 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 Processor -- start 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c0a2510 con 0x55790c1b64f0 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c0a21b0 con 0x55790c1b7810 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790bf89650 con 0x55790c1b6e50 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f93373c8b80 10 client.0 _ll_register_callbacks cb 0x55790c0abae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.274+0000 7f9328faa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1bfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36668/0 (socket says 172.21.15.64:36668) 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 -- 172.21.15.64:0/4053884335 learned_addr learned my addr 172.21.15.64:0/4053884335 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f93297ab700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9323fff700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 -- 172.21.15.64:0/4053884335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 msgr2=0x55790c1c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1c0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 -- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 msgr2=0x55790c1c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 -- 172.21.15.64:0/4053884335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931800fa10 con 0x55790c1b64f0 2024-05-08T21:54:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9328faa700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1bfde0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9318011b80 tx=0x7f9318011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e482837057000ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9303fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9303fff700 1 -- 172.21.15.64:0/4053884335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931802e070 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9303fff700 1 -- 172.21.15.64:0/4053884335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f931801e720 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55790c05f220 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f9303fff700 1 -- 172.21.15.64:0/4053884335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f931801e8e0 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f93373c8b80 10 client.15057 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.275+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55790bf33b10 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f9303fff700 1 -- 172.21.15.64:0/4053884335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55790c05f220 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f9303fff700 1 -- 172.21.15.64:0/4053884335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9318004b70 con 0x55790c1b64f0 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f93373c8b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f93373c8b80 10 client.15057 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f93373c8b80 20 client.15057 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f93373c8b80 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c1cad80 0x55790c1d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f93373c8b80 4 client.15057 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 --> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55790c1857c0 con 0x55790c1cad80 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.276+0000 7f9323fff700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c1cad80 0x55790c1d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f9303fff700 10 client.15057 ms_handle_connect on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f9323fff700 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c1cad80 0x55790c1d0020 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5eb6ccaaeb959c2 server_cookie=b8ebfab735852a6b in_seq=0 out_seq=0 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f9303fff700 1 -- 172.21.15.64:0/4053884335 <== mds.4 v2:172.21.15.160:6832/3542328594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55790c1857c0 con 0x55790c1cad80 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f9303fff700 10 client.15057 handle_command_reply: tid=0 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 client.15057 shutdown 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 msgr2=0x55790c1bfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1bfde0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9318011b80 tx=0x7f9318011bb0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 shutdown_connections 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c1cad80 0x55790c1d0020 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b7810 0x55790c1c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1bfde0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f9303fff700 0 client.15057 ms_handle_reset on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 --2- 172.21.15.64:0/4053884335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b6e50 0x55790c1c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.277+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 >> 172.21.15.64:0/4053884335 conn(0x55790c1141c0 msgr2=0x55790c1b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.278+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 shutdown_connections 2024-05-08T21:54:14.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.278+0000 7f93373c8b80 1 -- 172.21.15.64:0/4053884335 wait complete. 2024-05-08T21:54:14.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.278+0000 7f93373c8b80 20 client.15057 trim_cache size 0 max 16384 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 Processor -- start 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 -- start start 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b6e50 0x55790c1c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c0ff190 con 0x55790c1b64f0 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c103040 con 0x55790c1b6e50 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f93373c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c1be6c0 con 0x55790c1b7810 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36674/0 (socket says 172.21.15.64:36674) 2024-05-08T21:54:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 -- 172.21.15.64:0/2242467791 learned_addr learned my addr 172.21.15.64:0/2242467791 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9323fff700 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b6e50 0x55790c1c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 -- 172.21.15.64:0/2242467791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 msgr2=0x55790c1c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 -- 172.21.15.64:0/2242467791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b6e50 msgr2=0x55790c1c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b6e50 0x55790c1c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 -- 172.21.15.64:0/2242467791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55790c0cc510 con 0x55790c1b64f0 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9323fff700 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b6e50 0x55790c1c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f9328faa700 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b68d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9318004f30 tx=0x7f931800f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab28c38f56113e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.296+0000 7f930f7fe700 1 -- 172.21.15.64:0/2242467791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931802e040 con 0x55790c1b64f0 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f930f7fe700 1 -- 172.21.15.64:0/2242467791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931803f5a0 con 0x55790c1b64f0 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f930f7fe700 1 -- 172.21.15.64:0/2242467791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318004710 con 0x55790c1b64f0 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 -- 172.21.15.64:0/2242467791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 msgr2=0x55790c1b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b68d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9318004f30 tx=0x7f931800f8d0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 -- 172.21.15.64:0/2242467791 shutdown_connections 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b64f0 0x55790c1b68d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 --2- 172.21.15.64:0/2242467791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b6e50 0x55790c1c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 -- 172.21.15.64:0/2242467791 >> 172.21.15.64:0/2242467791 conn(0x55790c1141c0 msgr2=0x55790c1ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 -- 172.21.15.64:0/2242467791 shutdown_connections 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 -- 172.21.15.64:0/2242467791 wait complete. 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.297+0000 7f93373c8b80 1 -- start start 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 Processor -- start 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b64f0 0x55790c1c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 0x55790c1c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c1c72d0 con 0x55790c1b6e50 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c1c7440 con 0x55790c1b64f0 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790c1c7600 con 0x55790c1b7810 2024-05-08T21:54:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93373c8b80 10 client.0 _ll_register_callbacks cb 0x55790c0abae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 0x55790c1c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 0x55790c1c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36690/0 (socket says 172.21.15.64:36690) 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 -- 172.21.15.64:0/1520672143 learned_addr learned my addr 172.21.15.64:0/1520672143 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f93297ab700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9328faa700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b64f0 0x55790c1c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 -- 172.21.15.64:0/1520672143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 msgr2=0x55790c1c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 -- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b64f0 msgr2=0x55790c1c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b64f0 0x55790c1c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.298+0000 7f9323fff700 1 -- 172.21.15.64:0/1520672143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55790bf33b10 con 0x55790c1b6e50 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9323fff700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 0x55790c1c6850 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9310005f90 tx=0x7f931000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9fad88fbf6aa35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9303fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9303fff700 1 -- 172.21.15.64:0/1520672143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310016040 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9303fff700 1 -- 172.21.15.64:0/1520672143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9310003ac0 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9318011cb0 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9303fff700 1 -- 172.21.15.64:0/1520672143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f931001c8e0 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 10 client.15063 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55790bf8bc80 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9303fff700 1 -- 172.21.15.64:0/1520672143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9318011cb0 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f9303fff700 1 -- 172.21.15.64:0/1520672143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9310011c40 con 0x55790c1b6e50 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 10 client.15063 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 20 client.15063 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c2267b0 0x55790c223160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 4 client.15063 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.299+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 --> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55790c0fee70 con 0x55790c2267b0 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f9328faa700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c2267b0 0x55790c223160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f9303fff700 10 client.15063 ms_handle_connect on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f9328faa700 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c2267b0 0x55790c223160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccc71eea1664102e server_cookie=33636c4d3b29aeea in_seq=0 out_seq=0 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f9303fff700 1 -- 172.21.15.64:0/1520672143 <== mds.4 v2:172.21.15.160:6832/3542328594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55790c0fee70 con 0x55790c2267b0 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f9303fff700 10 client.15063 handle_command_reply: tid=0 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 client.15063 shutdown 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 msgr2=0x55790c1c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 0x55790c1c6850 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9310005f90 tx=0x7f931000bd40 comp rx=0 tx=0).stop 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 shutdown_connections 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x55790c2267b0 0x55790c223160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55790c1b64f0 0x55790c1c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f9303fff700 0 client.15063 ms_handle_reset on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55790c1b7810 0x55790c1c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 --2- 172.21.15.64:0/1520672143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55790c1b6e50 0x55790c1c6850 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 >> 172.21.15.64:0/1520672143 conn(0x55790c1141c0 msgr2=0x55790c1ba780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 shutdown_connections 2024-05-08T21:54:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.300+0000 7f93373c8b80 1 -- 172.21.15.64:0/1520672143 wait complete. 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f93373c8b80 20 client.15063 trim_cache size 0 max 16384 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 4, 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24422, 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 45.348315030000002, 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-05-08T21:54:14.310 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:14.310 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 74.456887926999997 2024-05-08T21:54:14.310 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:14.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f93140a1230 msgr2=0x7f93140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f93140a1230 0x7f93140a36f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9318005950 tx=0x7f93180058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 msgr2=0x7f932419e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f932419e520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f931c004d00 tx=0x7f931c004de0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 shutdown_connections 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f93140a1230 0x7f93140a36f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9324102c50 0x7f932419e520 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93240ff240 0x7f9324196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 --2- 172.21.15.64:0/3852282575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93241022b0 0x7f9324197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 >> 172.21.15.64:0/3852282575 conn(0x7f93240fab50 msgr2=0x7f9324105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 shutdown_connections 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.301+0000 7f932b20e700 1 -- 172.21.15.64:0/3852282575 wait complete. 2024-05-08T21:54:14.315 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-08T21:54:14.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 -- 172.21.15.64:0/1425001704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10106c60 msgr2=0x7f4e101090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 --2- 172.21.15.64:0/1425001704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10106c60 0x7f4e101090d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800b0a0 tx=0x7f4e0800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 -- 172.21.15.64:0/1425001704 shutdown_connections 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 --2- 172.21.15.64:0/1425001704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e10101a00 0x7f4e10103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 --2- 172.21.15.64:0/1425001704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10106c60 0x7f4e101090d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 --2- 172.21.15.64:0/1425001704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e10104330 0x7f4e10106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 -- 172.21.15.64:0/1425001704 >> 172.21.15.64:0/1425001704 conn(0x7f4e100fb3f0 msgr2=0x7f4e100fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 -- 172.21.15.64:0/1425001704 shutdown_connections 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.441+0000 7f4e161a0700 1 -- 172.21.15.64:0/1425001704 wait complete. 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 Processor -- start 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 -- start start 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 0x7f4e10196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e10104330 0x7f4e101974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e10106c60 0x7f4e1019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1019ead0 con 0x7f4e10101a00 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1019ec10 con 0x7f4e10104330 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e161a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1019eda0 con 0x7f4e10106c60 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 0x7f4e10196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 0x7f4e10196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36704/0 (socket says 172.21.15.64:36704) 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e0f7fe700 1 -- 172.21.15.64:0/741550878 learned_addr learned my addr 172.21.15.64:0/741550878 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.442+0000 7f4e0effd700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e10104330 0x7f4e101974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e10106c60 0x7f4e1019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0f7fe700 1 -- 172.21.15.64:0/741550878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e10106c60 msgr2=0x7f4e1019e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e10106c60 0x7f4e1019e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0f7fe700 1 -- 172.21.15.64:0/741550878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e10104330 msgr2=0x7f4e101974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e10104330 0x7f4e101974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0f7fe700 1 -- 172.21.15.64:0/741550878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e08009d00 con 0x7f4e10101a00 2024-05-08T21:54:14.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 0x7f4e10196fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000e9a0 tx=0x7f4e0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0cff9700 1 -- 172.21.15.64:0/741550878 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00009790 con 0x7f4e10101a00 2024-05-08T21:54:14.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0cff9700 1 -- 172.21.15.64:0/741550878 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0000cbd0 con 0x7f4e10101a00 2024-05-08T21:54:14.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1019f080 con 0x7f4e10101a00 2024-05-08T21:54:14.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e0cff9700 1 -- 172.21.15.64:0/741550878 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00018680 con 0x7f4e10101a00 2024-05-08T21:54:14.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.443+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1019f590 con 0x7f4e10101a00 2024-05-08T21:54:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.446+0000 7f4e2235ab80 1 Processor -- start 2024-05-08T21:54:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.446+0000 7f4e2235ab80 1 -- start start 2024-05-08T21:54:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459b4f0 0x55b93459b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b93459c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b93459cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9344f2100 con 0x55b93459b4f0 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9344efac0 con 0x55b93459c810 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9344eb3b0 con 0x55b93459be50 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b93459c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b93459c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:34672/0 (socket says 172.21.15.64:34672) 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0f7fe700 1 -- 172.21.15.64:0/4144981139 learned_addr learned my addr 172.21.15.64:0/4144981139 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0effd700 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b93459cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459b4f0 0x55b93459b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0effd700 1 -- 172.21.15.64:0/4144981139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 msgr2=0x55b93459c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0effd700 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b93459c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0effd700 1 -- 172.21.15.64:0/4144981139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459b4f0 msgr2=0x55b93459b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0effd700 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459b4f0 0x55b93459b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0effd700 1 -- 172.21.15.64:0/4144981139 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b93436c360 con 0x55b93459c810 2024-05-08T21:54:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.447+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459b4f0 0x55b93459b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e0effd700 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b93459cc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400ac00 tx=0x7f4e0400af10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbb9eb2f074371d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4df5ffb700 1 -- 172.21.15.64:0/4144981139 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400fd60 con 0x55b93459c810 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4df5ffb700 1 -- 172.21.15.64:0/4144981139 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0400fec0 con 0x55b93459c810 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4df5ffb700 1 -- 172.21.15.64:0/4144981139 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04018440 con 0x55b93459c810 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e0cff9700 1 -- 172.21.15.64:0/741550878 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4e0000cd40 con 0x7f4e10101a00 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 -- 172.21.15.64:0/4144981139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 msgr2=0x55b93459cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b93459cc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400ac00 tx=0x7f4e0400af10 comp rx=0 tx=0).stop 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 -- 172.21.15.64:0/4144981139 shutdown_connections 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b93459cc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459b4f0 0x55b93459b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/4144981139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b93459c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.448+0000 7f4e2235ab80 1 -- 172.21.15.64:0/4144981139 >> 172.21.15.64:0/4144981139 conn(0x55b9344f91c0 msgr2=0x55b934595d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e2235ab80 1 -- 172.21.15.64:0/4144981139 shutdown_connections 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e2235ab80 1 -- 172.21.15.64:0/4144981139 wait complete. 2024-05-08T21:54:14.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e2235ab80 1 -- start start 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e0cff9700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4df80a10c0 0x7f4df80a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e2235ab80 1 Processor -- start 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4df80a10c0 0x7f4df80a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.449+0000 7f4e0cff9700 1 -- 172.21.15.64:0/741550878 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4e00014070 con 0x7f4e10101a00 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459b4f0 0x55b9345a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b9345a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 0x55b9345ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b934487510 con 0x55b93459c810 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9344871b0 con 0x55b93459b4f0 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b93436e650 con 0x55b93459be50 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e2235ab80 10 client.0 _ll_register_callbacks cb 0x55b934490ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 0x55b9345ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 0x55b9345ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36722/0 (socket says 172.21.15.64:36722) 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 -- 172.21.15.64:0/3778744097 learned_addr learned my addr 172.21.15.64:0/3778744097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459b4f0 0x55b9345a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0effd700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b9345a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4df80a10c0 0x7f4df80a3580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4e080062a0 tx=0x7f4e080061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 -- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 msgr2=0x55b9345a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b9345a52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 -- 172.21.15.64:0/3778744097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459b4f0 msgr2=0x55b9345a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459b4f0 0x55b9345a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 -- 172.21.15.64:0/3778744097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0400a830 con 0x55b93459c810 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 0x55b9345ac2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e08015000 tx=0x7f4e0800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62edd9194eda39f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.450+0000 7f4deeffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4deeffd700 1 -- 172.21.15.64:0/3778744097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800e040 con 0x55b93459c810 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4deeffd700 1 -- 172.21.15.64:0/3778744097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4e08024040 con 0x55b93459c810 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b93456a7c0 con 0x55b93459c810 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4deeffd700 1 -- 172.21.15.64:0/3778744097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e0801cdb0 con 0x55b93459c810 2024-05-08T21:54:14.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 10 client.15072 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b934318b10 con 0x55b93459c810 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4deeffd700 1 -- 172.21.15.64:0/3778744097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b93456a7c0 con 0x55b93459c810 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4deeffd700 1 -- 172.21.15.64:0/3778744097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e080057c0 con 0x55b93459c810 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 10 client.15072 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 20 client.15072 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b9345afcb0 0x55b9345b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 4 client.15072 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 --> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9344e3e70 con 0x55b9345afcb0 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.451+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b9345afcb0 0x55b9345b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.452+0000 7f4deeffd700 10 client.15072 ms_handle_connect on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.452+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b9345afcb0 0x55b9345b4fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a298370ee438036 server_cookie=7766dbcb26366e0c in_seq=0 out_seq=0 2024-05-08T21:54:14.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4deeffd700 1 -- 172.21.15.64:0/3778744097 <== mds.3 v2:172.21.15.186:6832/1716244539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9344e3e70 con 0x55b9345afcb0 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4deeffd700 10 client.15072 handle_command_reply: tid=0 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 client.15072 shutdown 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 msgr2=0x55b9345ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 0x55b9345ac2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e08015000 tx=0x7f4e0800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 shutdown_connections 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b9345afcb0 0x55b9345b4fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459b4f0 0x55b9345a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4deeffd700 0 client.15072 ms_handle_reset on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459c810 0x55b9345ac2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/3778744097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459be50 0x55b9345a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 >> 172.21.15.64:0/3778744097 conn(0x55b9344f91c0 msgr2=0x55b934595550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 shutdown_connections 2024-05-08T21:54:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 1 -- 172.21.15.64:0/3778744097 wait complete. 2024-05-08T21:54:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.453+0000 7f4e2235ab80 20 client.15072 trim_cache size 0 max 16384 2024-05-08T21:54:14.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:14 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3503709215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:14 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/695192384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:14 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1218209243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 Processor -- start 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 -- start start 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b93459b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b93448a5a0 con 0x55b93459be50 2024-05-08T21:54:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9345a48e0 con 0x55b93459c810 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b93458e080 con 0x55b93459b4f0 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.471+0000 7f4e0effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b93459b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b93459b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36724/0 (socket says 172.21.15.64:36724) 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 -- 172.21.15.64:0/2824291151 learned_addr learned my addr 172.21.15.64:0/2824291151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 -- 172.21.15.64:0/2824291151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 msgr2=0x55b9345a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 -- 172.21.15.64:0/2824291151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 msgr2=0x55b9345a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 -- 172.21.15.64:0/2824291151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9344b1510 con 0x55b93459be50 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4e0effd700 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b93459b8d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400ab20 tx=0x7f4e04003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be495d798c0d43a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4df67fc700 1 -- 172.21.15.64:0/2824291151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400fd60 con 0x55b93459be50 2024-05-08T21:54:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4df67fc700 1 -- 172.21.15.64:0/2824291151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0400fec0 con 0x55b93459be50 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.472+0000 7f4df67fc700 1 -- 172.21.15.64:0/2824291151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004df0 con 0x55b93459be50 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 -- 172.21.15.64:0/2824291151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 msgr2=0x55b93459b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b93459b8d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400ab20 tx=0x7f4e04003b10 comp rx=0 tx=0).stop 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 -- 172.21.15.64:0/2824291151 shutdown_connections 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/2824291151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b93459b8d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 -- 172.21.15.64:0/2824291151 >> 172.21.15.64:0/2824291151 conn(0x55b9344f91c0 msgr2=0x55b93459cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 -- 172.21.15.64:0/2824291151 shutdown_connections 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 -- 172.21.15.64:0/2824291151 wait complete. 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 -- start start 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.473+0000 7f4e2235ab80 1 Processor -- start 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b9345a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9345ac420 con 0x55b93459be50 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9345ac590 con 0x55b93459c810 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9345ac750 con 0x55b93459b4f0 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 10 client.0 _ll_register_callbacks cb 0x55b934490ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b9345a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b9345a6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36736/0 (socket says 172.21.15.64:36736) 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 -- 172.21.15.64:0/1744353074 learned_addr learned my addr 172.21.15.64:0/1744353074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 -- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 msgr2=0x55b9345a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 -- 172.21.15.64:0/1744353074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 msgr2=0x55b9345abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 -- 172.21.15.64:0/1744353074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b93436c360 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0ffff700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e0effd700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b9345a6f50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005950 tx=0x7f4e0400a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458c93900a26a590 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4deeffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4deeffd700 1 -- 172.21.15.64:0/1744353074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04005450 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4deeffd700 1 -- 172.21.15.64:0/1744353074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4e040055b0 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.474+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b934339ca0 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4deeffd700 1 -- 172.21.15.64:0/1744353074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e04005770 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 10 client.15078 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b934370c80 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4deeffd700 1 -- 172.21.15.64:0/1744353074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b934339ca0 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4deeffd700 1 -- 172.21.15.64:0/1744353074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e04029020 con 0x55b93459be50 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 10 client.15078 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 20 client.15078 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b93460b430 0x55b934607da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 4 client.15078 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:14.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 --> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b93436ec90 con 0x55b93460b430 2024-05-08T21:54:14.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.475+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b93460b430 0x55b934607da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e0f7fe700 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b93460b430 0x55b934607da0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7daa67b88681bfa server_cookie=8c5934bfb7c9995c in_seq=0 out_seq=0 2024-05-08T21:54:14.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4deeffd700 10 client.15078 ms_handle_connect on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4deeffd700 1 -- 172.21.15.64:0/1744353074 <== mds.3 v2:172.21.15.186:6832/1716244539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b93436ec90 con 0x55b93460b430 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4deeffd700 10 client.15078 handle_command_reply: tid=0 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 client.15078 shutdown 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 msgr2=0x55b9345a6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b9345a6f50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005950 tx=0x7f4e0400a300 comp rx=0 tx=0).stop 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 shutdown_connections 2024-05-08T21:54:14.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x55b93460b430 0x55b934607da0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b93459c810 0x55b9345abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b93459b4f0 0x55b9345a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 --2- 172.21.15.64:0/1744353074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b93459be50 0x55b9345a6f50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4deeffd700 0 client.15078 ms_handle_reset on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 >> 172.21.15.64:0/1744353074 conn(0x55b9344f91c0 msgr2=0x55b9345a2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 shutdown_connections 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 1 -- 172.21.15.64:0/1744353074 wait complete. 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.476+0000 7f4e2235ab80 20 client.15078 trim_cache size 0 max 16384 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:14.484 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 3, 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24427, 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 46.533996305000002, 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 76.414366525999995 2024-05-08T21:54:14.485 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4df80a10c0 msgr2=0x7f4df80a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4df80a10c0 0x7f4df80a3580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4e080062a0 tx=0x7f4e080061f0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 msgr2=0x7f4e10196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 0x7f4e10196fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000e9a0 tx=0x7f4e0000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 shutdown_connections 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4df80a10c0 0x7f4df80a3580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e10106c60 0x7f4e1019e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e10101a00 0x7f4e10196fa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 --2- 172.21.15.64:0/741550878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e10104330 0x7f4e101974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 >> 172.21.15.64:0/741550878 conn(0x7f4e100fb3f0 msgr2=0x7f4e10102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 shutdown_connections 2024-05-08T21:54:14.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.477+0000 7f4e161a0700 1 -- 172.21.15.64:0/741550878 wait complete. 2024-05-08T21:54:14.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T21:54:14.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:14 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3503709215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:14 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/695192384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:14 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1218209243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.616+0000 7fd839368700 1 -- 172.21.15.64:0/1617400307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834101420 msgr2=0x7fd834101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.616+0000 7fd839368700 1 --2- 172.21.15.64:0/1617400307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834101420 0x7fd834101840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd828009990 tx=0x7fd828009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 -- 172.21.15.64:0/1617400307 shutdown_connections 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 --2- 172.21.15.64:0/1617400307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd834100af0 0x7fd8341083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 --2- 172.21.15.64:0/1617400307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834101420 0x7fd834101840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 --2- 172.21.15.64:0/1617400307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834100150 0x7fd8341005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 -- 172.21.15.64:0/1617400307 >> 172.21.15.64:0/1617400307 conn(0x7fd8340fab80 msgr2=0x7fd8340fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 -- 172.21.15.64:0/1617400307 shutdown_connections 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 -- 172.21.15.64:0/1617400307 wait complete. 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 Processor -- start 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.617+0000 7fd839368700 1 -- start start 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 0x7fd834103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd834100af0 0x7fd834102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834101420 0x7fd834102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834103220 con 0x7fd834100150 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834103390 con 0x7fd834101420 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341a0fc0 con 0x7fd834100af0 2024-05-08T21:54:14.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 0x7fd834103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 0x7fd834103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36756/0 (socket says 172.21.15.64:36756) 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 -- 172.21.15.64:0/3816984934 learned_addr learned my addr 172.21.15.64:0/3816984934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd834100af0 0x7fd834102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd8337fe700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834101420 0x7fd834102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 -- 172.21.15.64:0/3816984934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd834100af0 msgr2=0x7fd834102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd834100af0 0x7fd834102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 -- 172.21.15.64:0/3816984934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834101420 msgr2=0x7fd834102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834101420 0x7fd834102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 -- 172.21.15.64:0/3816984934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280095f0 con 0x7fd834100150 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd832ffd700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 0x7fd834103eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098e0 tx=0x7fd828005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd81bfff700 1 -- 172.21.15.64:0/3816984934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82801d070 con 0x7fd834100150 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd81bfff700 1 -- 172.21.15.64:0/3816984934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd828022470 con 0x7fd834100150 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8341a11f0 con 0x7fd834100150 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.618+0000 7fd81bfff700 1 -- 172.21.15.64:0/3816984934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828017650 con 0x7fd834100150 2024-05-08T21:54:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.619+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8341a15b0 con 0x7fd834100150 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.621+0000 7fd81bfff700 1 -- 172.21.15.64:0/3816984934 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd828022850 con 0x7fd834100150 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.622+0000 7fd81bfff700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd81c0a11b0 0x7fd81c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.622+0000 7fd81bfff700 1 -- 172.21.15.64:0/3816984934 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd8280cdef0 con 0x7fd834100150 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.622+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd81c0a11b0 0x7fd81c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.622+0000 7fd845522b80 1 Processor -- start 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.622+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd81c0a11b0 0x7fd81c0a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd824009c60 tx=0x7fd824009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.622+0000 7fd845522b80 1 -- start start 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd845522b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfac4f0 0x562d3cfac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd845522b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 0x562d3cfad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd845522b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfad810 0x562d3cfadc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd845522b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cf03100 con 0x562d3cface50 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd845522b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cf00ac0 con 0x562d3cfad810 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd845522b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cefc3b0 con 0x562d3cfac4f0 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 0x562d3cfad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 0x562d3cfad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36770/0 (socket says 172.21.15.64:36770) 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 -- 172.21.15.64:0/1795648251 learned_addr learned my addr 172.21.15.64:0/1795648251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd832ffd700 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfac4f0 0x562d3cfac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 -- 172.21.15.64:0/1795648251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfac4f0 msgr2=0x562d3cfac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfac4f0 0x562d3cfac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 -- 172.21.15.64:0/1795648251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfad810 msgr2=0x562d3cfadc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfad810 0x562d3cfadc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 -- 172.21.15.64:0/1795648251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d3cd7d360 con 0x562d3cface50 2024-05-08T21:54:14.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd832ffd700 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfac4f0 0x562d3cfac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8327fc700 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 0x562d3cfad2d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e4e0 tx=0x7fd824014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674bc11f9fbaa3ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8197fa700 1 -- 172.21.15.64:0/1795648251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824026430 con 0x562d3cface50 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8197fa700 1 -- 172.21.15.64:0/1795648251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd824021070 con 0x562d3cface50 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.623+0000 7fd8197fa700 1 -- 172.21.15.64:0/1795648251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82401d7d0 con 0x562d3cface50 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 -- 172.21.15.64:0/1795648251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 msgr2=0x562d3cfad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 0x562d3cfad2d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e4e0 tx=0x7fd824014000 comp rx=0 tx=0).stop 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 -- 172.21.15.64:0/1795648251 shutdown_connections 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfad810 0x562d3cfadc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfac4f0 0x562d3cfac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 --2- 172.21.15.64:0/1795648251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cface50 0x562d3cfad2d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 -- 172.21.15.64:0/1795648251 >> 172.21.15.64:0/1795648251 conn(0x562d3cf0a1c0 msgr2=0x562d3cfa6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 -- 172.21.15.64:0/1795648251 shutdown_connections 2024-05-08T21:54:14.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 -- 172.21.15.64:0/1795648251 wait complete. 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.624+0000 7fd845522b80 1 -- start start 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 Processor -- start 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3ce98510 con 0x562d3cfac4f0 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3ce981b0 con 0x562d3cface50 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cd7f650 con 0x562d3cfad810 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 10 client.0 _ll_register_callbacks cb 0x562d3cea1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36774/0 (socket says 172.21.15.64:36774) 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 -- 172.21.15.64:0/4260025496 learned_addr learned my addr 172.21.15.64:0/4260025496 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd8327fc700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd8337fe700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 -- 172.21.15.64:0/4260025496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 msgr2=0x562d3cfbd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 -- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 msgr2=0x562d3cfb6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 -- 172.21.15.64:0/4260025496 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd824017070 con 0x562d3cfac4f0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd832ffd700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb5e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd82809ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4eb0bcd3332fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd816ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd816ff5700 1 -- 172.21.15.64:0/4260025496 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82809f170 con 0x562d3cfac4f0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd816ff5700 1 -- 172.21.15.64:0/4260025496 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd82809f380 con 0x562d3cfac4f0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.625+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d3ce55220 con 0x562d3cfac4f0 2024-05-08T21:54:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd816ff5700 1 -- 172.21.15.64:0/4260025496 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8280a3940 con 0x562d3cfac4f0 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 10 client.15090 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562d3cd29b10 con 0x562d3cfac4f0 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd816ff5700 1 -- 172.21.15.64:0/4260025496 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d3ce55220 con 0x562d3cfac4f0 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd816ff5700 1 -- 172.21.15.64:0/4260025496 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd828097460 con 0x562d3cfac4f0 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 20 client.15090 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3cfc0bc0 0x562d3cfc5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 4 client.15090 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 --> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d3cf7b7c0 con 0x562d3cfc0bc0 2024-05-08T21:54:14.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.626+0000 7fd8327fc700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3cfc0bc0 0x562d3cfc5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.627+0000 7fd816ff5700 10 client.15090 ms_handle_connect on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.627+0000 7fd8327fc700 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3cfc0bc0 0x562d3cfc5ec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1254987696790a35 server_cookie=460f8c74355f43fe in_seq=0 out_seq=0 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd816ff5700 1 -- 172.21.15.64:0/4260025496 <== mds.2 v2:172.21.15.160:6836/4233185014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d3cf7b7c0 con 0x562d3cfc0bc0 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd816ff5700 10 client.15090 handle_command_reply: tid=0 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 client.15090 shutdown 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 msgr2=0x562d3cfb5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb5e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd82809ba90 comp rx=0 tx=0).stop 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 shutdown_connections 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3cfc0bc0 0x562d3cfc5ec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd816ff5700 0 client.15090 ms_handle_reset on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb5e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 --2- 172.21.15.64:0/4260025496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 >> 172.21.15.64:0/4260025496 conn(0x562d3cf0a1c0 msgr2=0x562d3cfa6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 shutdown_connections 2024-05-08T21:54:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 1 -- 172.21.15.64:0/4260025496 wait complete. 2024-05-08T21:54:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.628+0000 7fd845522b80 20 client.15090 trim_cache size 0 max 16384 2024-05-08T21:54:14.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:14 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3503709215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:14 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/695192384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:14 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1218209243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 Processor -- start 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 -- start start 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfac4f0 0x562d3cfb6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cface50 0x562d3cfac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 0x562d3cfbda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cf7b7c0 con 0x562d3cfad810 2024-05-08T21:54:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3ce55220 con 0x562d3cfac4f0 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd845522b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3ceba2b0 con 0x562d3cface50 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.646+0000 7fd8337fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 0x562d3cfbda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 0x562d3cfbda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36776/0 (socket says 172.21.15.64:36776) 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 -- 172.21.15.64:0/4185201524 learned_addr learned my addr 172.21.15.64:0/4185201524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd832ffd700 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfac4f0 0x562d3cfb6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8327fc700 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cface50 0x562d3cfac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 -- 172.21.15.64:0/4185201524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cface50 msgr2=0x562d3cfac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cface50 0x562d3cfac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 -- 172.21.15.64:0/4185201524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfac4f0 msgr2=0x562d3cfb6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfac4f0 0x562d3cfb6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 -- 172.21.15.64:0/4185201524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d3cec2510 con 0x562d3cfad810 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd8337fe700 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 0x562d3cfbda50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009a20 tx=0x7fd82c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dcba575b29af61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd819ffb700 1 -- 172.21.15.64:0/4185201524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c019070 con 0x562d3cfad810 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd819ffb700 1 -- 172.21.15.64:0/4185201524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82c009e50 con 0x562d3cfad810 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.647+0000 7fd819ffb700 1 -- 172.21.15.64:0/4185201524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0055f0 con 0x562d3cfad810 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 -- 172.21.15.64:0/4185201524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 msgr2=0x562d3cfbda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 0x562d3cfbda50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009a20 tx=0x7fd82c009040 comp rx=0 tx=0).stop 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 -- 172.21.15.64:0/4185201524 shutdown_connections 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cfac4f0 0x562d3cfb6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfad810 0x562d3cfbda50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 --2- 172.21.15.64:0/4185201524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cface50 0x562d3cfac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 -- 172.21.15.64:0/4185201524 >> 172.21.15.64:0/4185201524 conn(0x562d3cf0a1c0 msgr2=0x562d3cfa31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 -- 172.21.15.64:0/4185201524 shutdown_connections 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 -- 172.21.15.64:0/4185201524 wait complete. 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 -- start start 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.648+0000 7fd845522b80 1 Processor -- start 2024-05-08T21:54:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cfbcae0 con 0x562d3cfac4f0 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cfbcc50 con 0x562d3cface50 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3cfbce10 con 0x562d3cfad810 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 10 client.0 _ll_register_callbacks cb 0x562d3cea1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36788/0 (socket says 172.21.15.64:36788) 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 -- 172.21.15.64:0/3707304361 learned_addr learned my addr 172.21.15.64:0/3707304361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd8337fe700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 -- 172.21.15.64:0/3707304361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 msgr2=0x562d3cfbc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 -- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 msgr2=0x562d3cfb7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 -- 172.21.15.64:0/3707304361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c011040 con 0x562d3cfac4f0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd832ffd700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb7550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b680 tx=0x7fd82809b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fae859876eab53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd816ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd816ff5700 1 -- 172.21.15.64:0/3707304361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82809f170 con 0x562d3cfac4f0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd816ff5700 1 -- 172.21.15.64:0/3707304361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd82809f380 con 0x562d3cfac4f0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd824021070 con 0x562d3cfac4f0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd816ff5700 1 -- 172.21.15.64:0/3707304361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82800fb70 con 0x562d3cfac4f0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 10 client.15096 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.649+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562d3cd81c80 con 0x562d3cfac4f0 2024-05-08T21:54:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd816ff5700 1 -- 172.21.15.64:0/3707304361 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd824021070 con 0x562d3cfac4f0 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd816ff5700 1 -- 172.21.15.64:0/3707304361 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd82809a5d0 con 0x562d3cfac4f0 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd845522b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd845522b80 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd845522b80 20 client.15096 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd845522b80 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3d01b870 0x562d3d0182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd845522b80 4 client.15096 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 --> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d3cef4e70 con 0x562d3d01b870 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3d01b870 0x562d3d0182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd816ff5700 10 client.15096 ms_handle_connect on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.650+0000 7fd8327fc700 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3d01b870 0x562d3d0182a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b0ff0e85ae3f04d server_cookie=72d78a31fe07467f in_seq=0 out_seq=0 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd816ff5700 1 -- 172.21.15.64:0/3707304361 <== mds.2 v2:172.21.15.160:6836/4233185014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562d3cef4e70 con 0x562d3d01b870 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd816ff5700 10 client.15096 handle_command_reply: tid=0 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 client.15096 shutdown 2024-05-08T21:54:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 msgr2=0x562d3cfb7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb7550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b680 tx=0x7fd82809b6d0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 shutdown_connections 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x562d3d01b870 0x562d3d0182a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d3cfad810 0x562d3cfbc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd816ff5700 0 client.15096 ms_handle_reset on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d3cfac4f0 0x562d3cfb7550 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 --2- 172.21.15.64:0/3707304361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d3cface50 0x562d3cfb7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 >> 172.21.15.64:0/3707304361 conn(0x562d3cf0a1c0 msgr2=0x562d3cfb5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 shutdown_connections 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 1 -- 172.21.15.64:0/3707304361 wait complete. 2024-05-08T21:54:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd845522b80 20 client.15096 trim_cache size 0 max 16384 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24476, 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 47.060638914999998, 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-05-08T21:54:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:14.661 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 64.316721955000006 2024-05-08T21:54:14.661 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd81c0a11b0 msgr2=0x7fd81c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd839368700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd81c0a11b0 0x7fd81c0a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd824009c60 tx=0x7fd824009260 comp rx=0 tx=0).stop 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 msgr2=0x7fd834103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.651+0000 7fd839368700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 0x7fd834103eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098e0 tx=0x7fd828005720 comp rx=0 tx=0).stop 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 shutdown_connections 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd81c0a11b0 0x7fd81c0a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834101420 0x7fd834102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd834100150 0x7fd834103eb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 --2- 172.21.15.64:0/3816984934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd834100af0 0x7fd834102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 >> 172.21.15.64:0/3816984934 conn(0x7fd8340fab80 msgr2=0x7fd8340ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 shutdown_connections 2024-05-08T21:54:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.652+0000 7fd839368700 1 -- 172.21.15.64:0/3816984934 wait complete. 2024-05-08T21:54:14.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T21:54:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 -- 172.21.15.64:0/1223132218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d201020f0 msgr2=0x7f8d20102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 --2- 172.21.15.64:0/1223132218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d201020f0 0x7f8d20102570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009990 tx=0x7f8d10009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 -- 172.21.15.64:0/1223132218 shutdown_connections 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 --2- 172.21.15.64:0/1223132218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d20100f20 0x7f8d20101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 --2- 172.21.15.64:0/1223132218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d20102ab0 0x7f8d201093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 --2- 172.21.15.64:0/1223132218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d201020f0 0x7f8d20102570 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.791+0000 7f8d2822c700 1 -- 172.21.15.64:0/1223132218 >> 172.21.15.64:0/1223132218 conn(0x7f8d200fc4e0 msgr2=0x7f8d200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 -- 172.21.15.64:0/1223132218 shutdown_connections 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 -- 172.21.15.64:0/1223132218 wait complete. 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 Processor -- start 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 -- start start 2024-05-08T21:54:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d20100f20 0x7f8d20074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d201020f0 0x7f8d20073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 0x7f8d200738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20074070 con 0x7f8d20102ab0 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200741e0 con 0x7f8d20100f20 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d2822c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201a0f90 con 0x7f8d201020f0 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.792+0000 7f8d267c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 0x7f8d200738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 0x7f8d200738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36800/0 (socket says 172.21.15.64:36800) 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 -- 172.21.15.64:0/2341919212 learned_addr learned my addr 172.21.15.64:0/2341919212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d20100f20 0x7f8d20074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d257c7700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d201020f0 0x7f8d20073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 -- 172.21.15.64:0/2341919212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d201020f0 msgr2=0x7f8d20073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d201020f0 0x7f8d20073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 -- 172.21.15.64:0/2341919212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d20100f20 msgr2=0x7f8d20074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d20100f20 0x7f8d20074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 -- 172.21.15.64:0/2341919212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100095f0 con 0x7f8d20102ab0 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d267c9700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 0x7f8d200738d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8d18007ee0 tx=0x7f8d1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d16ffd700 1 -- 172.21.15.64:0/2341919212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1800dd70 con 0x7f8d20102ab0 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d16ffd700 1 -- 172.21.15.64:0/2341919212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1800f040 con 0x7f8d20102ab0 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d201a1220 con 0x7f8d20102ab0 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d16ffd700 1 -- 172.21.15.64:0/2341919212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d180048a0 con 0x7f8d20102ab0 2024-05-08T21:54:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.793+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d201a1730 con 0x7f8d20102ab0 2024-05-08T21:54:14.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.795+0000 7f8d16ffd700 1 -- 172.21.15.64:0/2341919212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8d18003a00 con 0x7f8d20102ab0 2024-05-08T21:54:14.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.796+0000 7f8d16ffd700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8d0c0a1230 0x7f8d0c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.796+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8d0c0a1230 0x7f8d0c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.796+0000 7f8d16ffd700 1 -- 172.21.15.64:0/2341919212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8d180cc1e0 con 0x7f8d20102ab0 2024-05-08T21:54:14.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.796+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8d0c0a1230 0x7f8d0c0a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d20100c40 tx=0x7f8d1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 Processor -- start 2024-05-08T21:54:14.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 -- start start 2024-05-08T21:54:14.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbd4f0 0x56044bcbd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcbe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbe810 0x56044bcbec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bc14100 con 0x56044bcbde50 2024-05-08T21:54:14.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bc11ac0 con 0x56044bcbe810 2024-05-08T21:54:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bc0d3b0 con 0x56044bcbd4f0 2024-05-08T21:54:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d257c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcbe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d257c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcbe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36802/0 (socket says 172.21.15.64:36802) 2024-05-08T21:54:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d257c7700 1 -- 172.21.15.64:0/1227322511 learned_addr learned my addr 172.21.15.64:0/1227322511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d267c9700 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbe810 0x56044bcbec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.797+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbd4f0 0x56044bcbd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d257c7700 1 -- 172.21.15.64:0/1227322511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbd4f0 msgr2=0x56044bcbd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d257c7700 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbd4f0 0x56044bcbd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d257c7700 1 -- 172.21.15.64:0/1227322511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbe810 msgr2=0x56044bcbec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d257c7700 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbe810 0x56044bcbec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d257c7700 1 -- 172.21.15.64:0/1227322511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044ba8e360 con 0x56044bcbde50 2024-05-08T21:54:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d267c9700 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbe810 0x56044bcbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d257c7700 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcbe2d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009580 tx=0x7f8d10005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f5dc8ad0ef4aa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d0bfff700 1 -- 172.21.15.64:0/1227322511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10005240 con 0x56044bcbde50 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d0bfff700 1 -- 172.21.15.64:0/1227322511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1001e040 con 0x56044bcbde50 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.798+0000 7f8d0bfff700 1 -- 172.21.15.64:0/1227322511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004760 con 0x56044bcbde50 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1227322511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 msgr2=0x56044bcbe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcbe2d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009580 tx=0x7f8d10005e80 comp rx=0 tx=0).stop 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1227322511 shutdown_connections 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbe810 0x56044bcbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbd4f0 0x56044bcbd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1227322511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcbe2d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1227322511 >> 172.21.15.64:0/1227322511 conn(0x56044bc1b1c0 msgr2=0x56044bcb7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1227322511 shutdown_connections 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1227322511 wait complete. 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 -- start start 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.799+0000 7f8d343e6b80 1 Processor -- start 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbde50 0x56044bcc7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 0x56044bcce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bba9510 con 0x56044bcbe810 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bba91b0 con 0x56044bcbd4f0 2024-05-08T21:54:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044ba90650 con 0x56044bcbde50 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 10 client.0 _ll_register_callbacks cb 0x56044bbb2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 0x56044bcce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 0x56044bcce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36810/0 (socket says 172.21.15.64:36810) 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 -- 172.21.15.64:0/1825708074 learned_addr learned my addr 172.21.15.64:0/1825708074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d257c7700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbde50 0x56044bcc7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 -- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbde50 msgr2=0x56044bcc7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbde50 0x56044bcc7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 -- 172.21.15.64:0/1825708074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 msgr2=0x56044bcc6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 -- 172.21.15.64:0/1825708074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d20109d00 con 0x56044bcbe810 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d267c9700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 0x56044bcce2d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d18095ff0 tx=0x7f8d18099c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54fe20fc9c95664 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d097fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d097fa700 1 -- 172.21.15.64:0/1825708074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1809d3b0 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d097fa700 1 -- 172.21.15.64:0/1825708074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8d18095040 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.800+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56044bb66220 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d097fa700 1 -- 172.21.15.64:0/1825708074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d180a1a00 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 10 client.15108 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56044ba3ab10 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d097fa700 1 -- 172.21.15.64:0/1825708074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d180b6030 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d097fa700 1 -- 172.21.15.64:0/1825708074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d180997e0 con 0x56044bcbe810 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 20 client.15108 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bcd1a00 0x56044bcd72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 4 client.15108 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 --> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56044bc8c7c0 con 0x56044bcd1a00 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.801+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bcd1a00 0x56044bcd72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.802+0000 7f8d097fa700 10 client.15108 ms_handle_connect on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.802+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bcd1a00 0x56044bcd72f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d77b48305faffcf server_cookie=fe83da7b0aee3168 in_seq=0 out_seq=0 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.802+0000 7f8d097fa700 1 -- 172.21.15.64:0/1825708074 <== mds.1 v2:172.21.15.186:6838/539285459 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56044bc8c7c0 con 0x56044bcd1a00 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.802+0000 7f8d097fa700 10 client.15108 handle_command_reply: tid=0 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.802+0000 7f8d343e6b80 1 client.15108 shutdown 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 msgr2=0x56044bcce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 0x56044bcce2d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d18095ff0 tx=0x7f8d18099c80 comp rx=0 tx=0).stop 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 shutdown_connections 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bcd1a00 0x56044bcd72f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbe810 0x56044bcce2d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d097fa700 0 client.15108 ms_handle_reset on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/1825708074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbde50 0x56044bcc7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 >> 172.21.15.64:0/1825708074 conn(0x56044bc1b1c0 msgr2=0x56044bcb7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 shutdown_connections 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 1 -- 172.21.15.64:0/1825708074 wait complete. 2024-05-08T21:54:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.803+0000 7f8d343e6b80 20 client.15108 trim_cache size 0 max 16384 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 Processor -- start 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 -- start start 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bcce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bc06150 con 0x56044bcbde50 2024-05-08T21:54:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bc0a010 con 0x56044bcbd4f0 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.821+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bcc68c0 con 0x56044bcbe810 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36816/0 (socket says 172.21.15.64:36816) 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 -- 172.21.15.64:0/4150635663 learned_addr learned my addr 172.21.15.64:0/4150635663 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 -- 172.21.15.64:0/4150635663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 msgr2=0x56044bcce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d267c9700 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bcce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bcce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 -- 172.21.15.64:0/4150635663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 msgr2=0x56044bcc7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 -- 172.21.15.64:0/4150635663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044bbd3510 con 0x56044bcbde50 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d257c7700 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcce290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009990 tx=0x7f8d10005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caab08121abb62e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d14ff9700 1 -- 172.21.15.64:0/4150635663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000b970 con 0x56044bcbde50 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d14ff9700 1 -- 172.21.15.64:0/4150635663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1000bad0 con 0x56044bcbde50 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.822+0000 7f8d14ff9700 1 -- 172.21.15.64:0/4150635663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004a60 con 0x56044bcbde50 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 -- 172.21.15.64:0/4150635663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 msgr2=0x56044bcce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcce290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009990 tx=0x7f8d10005900 comp rx=0 tx=0).stop 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 -- 172.21.15.64:0/4150635663 shutdown_connections 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bcce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/4150635663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcce290 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 -- 172.21.15.64:0/4150635663 >> 172.21.15.64:0/4150635663 conn(0x56044bc1b1c0 msgr2=0x56044bcbebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 -- 172.21.15.64:0/4150635663 shutdown_connections 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 -- 172.21.15.64:0/4150635663 wait complete. 2024-05-08T21:54:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 -- start start 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.823+0000 7f8d343e6b80 1 Processor -- start 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcc89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bccd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bccdb60 con 0x56044bcbde50 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bccdcd0 con 0x56044bcbd4f0 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044bccde90 con 0x56044bcbe810 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 10 client.0 _ll_register_callbacks cb 0x56044bbb2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcc89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcc89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36828/0 (socket says 172.21.15.64:36828) 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 -- 172.21.15.64:0/2504122914 learned_addr learned my addr 172.21.15.64:0/2504122914 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 -- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 msgr2=0x56044bccd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d267c9700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bccd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bccd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 -- 172.21.15.64:0/2504122914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 msgr2=0x56044bcc84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 -- 172.21.15.64:0/2504122914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56044ba8e360 con 0x56044bcbde50 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d257c7700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcc89f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005ec0 tx=0x7f8d10004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1116e44ca77176cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d097fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d097fa700 1 -- 172.21.15.64:0/2504122914 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10005240 con 0x56044bcbde50 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d097fa700 1 -- 172.21.15.64:0/2504122914 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8d100053a0 con 0x56044bcbde50 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.824+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d1001e040 con 0x56044bcbde50 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d097fa700 1 -- 172.21.15.64:0/2504122914 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d10005560 con 0x56044bcbde50 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 10 client.15114 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56044ba92c80 con 0x56044bcbde50 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d097fa700 1 -- 172.21.15.64:0/2504122914 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1001e040 con 0x56044bcbde50 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d097fa700 1 -- 172.21.15.64:0/2504122914 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d1003f710 con 0x56044bcbde50 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 20 client.15114 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bd2cf60 0x56044bd29910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 4 client.15114 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 --> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56044bc05e70 con 0x56044bd2cf60 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.825+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bd2cf60 0x56044bd29910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d097fa700 10 client.15114 ms_handle_connect on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d25fc8700 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bd2cf60 0x56044bd29910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fe091f921a054d server_cookie=6719ff6b1b3947ee in_seq=0 out_seq=0 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d097fa700 1 -- 172.21.15.64:0/2504122914 <== mds.1 v2:172.21.15.186:6838/539285459 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56044bc05e70 con 0x56044bd2cf60 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d097fa700 10 client.15114 handle_command_reply: tid=0 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 client.15114 shutdown 2024-05-08T21:54:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 msgr2=0x56044bcc89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcc89f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005ec0 tx=0x7f8d10004ac0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 shutdown_connections 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x56044bd2cf60 0x56044bd29910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56044bcbd4f0 0x56044bcc84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56044bcbe810 0x56044bccd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 --2- 172.21.15.64:0/2504122914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56044bcbde50 0x56044bcc89f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d097fa700 0 client.15114 ms_handle_reset on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 >> 172.21.15.64:0/2504122914 conn(0x56044bc1b1c0 msgr2=0x56044bcce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 shutdown_connections 2024-05-08T21:54:14.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 1 -- 172.21.15.64:0/2504122914 wait complete. 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.826+0000 7f8d343e6b80 20 client.15114 trim_cache size 0 max 16384 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24484, 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 48.242766144999997, 2024-05-08T21:54:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-05-08T21:54:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 62.749799953999997 2024-05-08T21:54:14.836 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8d0c0a1230 msgr2=0x7f8d0c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8d0c0a1230 0x7f8d0c0a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d20100c40 tx=0x7f8d1c00b410 comp rx=0 tx=0).stop 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 msgr2=0x7f8d200738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 0x7f8d200738d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8d18007ee0 tx=0x7f8d1800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 shutdown_connections 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f8d0c0a1230 0x7f8d0c0a36f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d20100f20 0x7f8d20074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d20102ab0 0x7f8d200738d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 --2- 172.21.15.64:0/2341919212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d201020f0 0x7f8d20073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 >> 172.21.15.64:0/2341919212 conn(0x7f8d200fc4e0 msgr2=0x7f8d20105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 shutdown_connections 2024-05-08T21:54:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.827+0000 7f8d2822c700 1 -- 172.21.15.64:0/2341919212 wait complete. 2024-05-08T21:54:14.841 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T21:54:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 -- 172.21.15.64:0/1514240290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 msgr2=0x7f1528100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1514240290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528100280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1510009990 tx=0x7f1510009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 -- 172.21.15.64:0/1514240290 shutdown_connections 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1514240290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1528101970 0x7f1528108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1514240290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528100280 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1514240290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 0x7f1528101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 -- 172.21.15.64:0/1514240290 >> 172.21.15.64:0/1514240290 conn(0x7f15280fb3f0 msgr2=0x7f15280fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 -- 172.21.15.64:0/1514240290 shutdown_connections 2024-05-08T21:54:14.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.967+0000 7f152cbfa700 1 -- 172.21.15.64:0/1514240290 wait complete. 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 Processor -- start 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 -- start start 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 0x7f1528196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1528101970 0x7f152819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152819d8f0 con 0x7f15280ffe60 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152819da60 con 0x7f1528101970 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152cbfa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528068070 con 0x7f1528100fb0 2024-05-08T21:54:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36850/0 (socket says 172.21.15.64:36850) 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 -- 172.21.15.64:0/1337368319 learned_addr learned my addr 172.21.15.64:0/1337368319 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f1526d9d700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1528101970 0x7f152819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f1525d9b700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 0x7f1528196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 -- 172.21.15.64:0/1337368319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 msgr2=0x7f1528196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 0x7f1528196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 -- 172.21.15.64:0/1337368319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1528101970 msgr2=0x7f152819d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1528101970 0x7f152819d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f152659c700 1 -- 172.21.15.64:0/1337368319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100095f0 con 0x7f15280ffe60 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.968+0000 7f1525d9b700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 0x7f1528196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.969+0000 7f152659c700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528195e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1510005910 tx=0x7f1510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.969+0000 7f151f7fe700 1 -- 172.21.15.64:0/1337368319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151001d070 con 0x7f15280ffe60 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.969+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280682f0 con 0x7f15280ffe60 2024-05-08T21:54:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.969+0000 7f151f7fe700 1 -- 172.21.15.64:0/1337368319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1510003f60 con 0x7f15280ffe60 2024-05-08T21:54:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.969+0000 7f151f7fe700 1 -- 172.21.15.64:0/1337368319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510022cc0 con 0x7f15280ffe60 2024-05-08T21:54:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.969+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280687a0 con 0x7f15280ffe60 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.971+0000 7f151f7fe700 1 -- 172.21.15.64:0/1337368319 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f15100040d0 con 0x7f15280ffe60 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.972+0000 7f151f7fe700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15140a11b0 0x7f15140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.972+0000 7f1525d9b700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15140a11b0 0x7f15140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.972+0000 7f151f7fe700 1 -- 172.21.15.64:0/1337368319 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f15100cdda0 con 0x7f15280ffe60 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.972+0000 7f1525d9b700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15140a11b0 0x7f15140a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1518005fd0 tx=0x7f1518005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.972+0000 7f1538db4b80 1 Processor -- start 2024-05-08T21:54:14.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 -- start start 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe24910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe24e50 0x55928fe252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe25810 0x55928fe25c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fd7b100 con 0x55928fe244f0 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fd78ac0 con 0x55928fe25810 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1538db4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fd743b0 con 0x55928fe24e50 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe24910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe24910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36854/0 (socket says 172.21.15.64:36854) 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 -- 172.21.15.64:0/3105884757 learned_addr learned my addr 172.21.15.64:0/3105884757 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1525d9b700 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe24e50 0x55928fe252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f1526d9d700 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe25810 0x55928fe25c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 -- 172.21.15.64:0/3105884757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe24e50 msgr2=0x55928fe252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe24e50 0x55928fe252d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 -- 172.21.15.64:0/3105884757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe25810 msgr2=0x55928fe25c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe25810 0x55928fe25c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 -- 172.21.15.64:0/3105884757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55928fbf5360 con 0x55928fe244f0 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.973+0000 7f152659c700 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe24910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f151000b5c0 tx=0x7f151009ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e10acf4395bb17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f151cff9700 1 -- 172.21.15.64:0/3105884757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151009f020 con 0x55928fe244f0 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f151cff9700 1 -- 172.21.15.64:0/3105884757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15100a1050 con 0x55928fe244f0 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f151cff9700 1 -- 172.21.15.64:0/3105884757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15100a3350 con 0x55928fe244f0 2024-05-08T21:54:14.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 -- 172.21.15.64:0/3105884757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 msgr2=0x55928fe24910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe24910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f151000b5c0 tx=0x7f151009ae00 comp rx=0 tx=0).stop 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 -- 172.21.15.64:0/3105884757 shutdown_connections 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe25810 0x55928fe25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe24910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3105884757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe24e50 0x55928fe252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.974+0000 7f1538db4b80 1 -- 172.21.15.64:0/3105884757 >> 172.21.15.64:0/3105884757 conn(0x55928fd821c0 msgr2=0x55928fe1ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 -- 172.21.15.64:0/3105884757 shutdown_connections 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 -- 172.21.15.64:0/3105884757 wait complete. 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 -- start start 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 Processor -- start 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe34f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fd10510 con 0x55928fe244f0 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fd101b0 con 0x55928fe24e50 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fbf7650 con 0x55928fe25810 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1538db4b80 10 client.0 _ll_register_callbacks cb 0x55928fd19ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36862/0 (socket says 172.21.15.64:36862) 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f152659c700 1 -- 172.21.15.64:0/2677507350 learned_addr learned my addr 172.21.15.64:0/2677507350 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1525d9b700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.975+0000 7f1526d9d700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe34f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f152659c700 1 -- 172.21.15.64:0/2677507350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 msgr2=0x55928fe34f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f152659c700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe34f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f152659c700 1 -- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 msgr2=0x55928fe2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f152659c700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f152659c700 1 -- 172.21.15.64:0/2677507350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280687a0 con 0x55928fe244f0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f152659c700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2d9f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1510000c00 tx=0x7f1510009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f282d8510e8dab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f150dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f150dffb700 1 -- 172.21.15.64:0/2677507350 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151009f220 con 0x55928fe244f0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f150dffb700 1 -- 172.21.15.64:0/2677507350 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f15100a37e0 con 0x55928fe244f0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55928fccd220 con 0x55928fe244f0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f150dffb700 1 -- 172.21.15.64:0/2677507350 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15100a3560 con 0x55928fe244f0 2024-05-08T21:54:14.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f1538db4b80 10 client.15126 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55928fba1b10 con 0x55928fe244f0 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f150dffb700 1 -- 172.21.15.64:0/2677507350 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55928fccd220 con 0x55928fe244f0 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.976+0000 7f150dffb700 1 -- 172.21.15.64:0/2677507350 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1510084020 con 0x55928fe244f0 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1538db4b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1538db4b80 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1538db4b80 20 client.15126 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe38700 0x55928fe3d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1538db4b80 4 client.15126 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55928fdf37c0 con 0x55928fe38700 2024-05-08T21:54:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1525d9b700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe38700 0x55928fe3d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f1525d9b700 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe38700 0x55928fe3d9a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435bb37065557a10 server_cookie=1a3a35e15e66543d in_seq=0 out_seq=0 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.977+0000 7f150dffb700 10 client.15126 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f150dffb700 1 -- 172.21.15.64:0/2677507350 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55928fdf37c0 con 0x55928fe38700 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f150dffb700 10 client.15126 handle_command_reply: tid=0 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 client.15126 shutdown 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 msgr2=0x55928fe2d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2d9f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1510000c00 tx=0x7f1510009f50 comp rx=0 tx=0).stop 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 shutdown_connections 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe38700 0x55928fe3d9a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe34f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2d9f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f150dffb700 0 client.15126 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2677507350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 >> 172.21.15.64:0/2677507350 conn(0x55928fd821c0 msgr2=0x55928fe1e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 shutdown_connections 2024-05-08T21:54:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 1 -- 172.21.15.64:0/2677507350 wait complete. 2024-05-08T21:54:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.978+0000 7f1538db4b80 20 client.15126 trim_cache size 0 max 16384 2024-05-08T21:54:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.996+0000 7f1538db4b80 1 Processor -- start 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.996+0000 7f1538db4b80 1 -- start start 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe25bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1538db4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fdf37c0 con 0x55928fe244f0 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1538db4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fccd220 con 0x55928fe24e50 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1538db4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fd322b0 con 0x55928fe25810 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36864/0 (socket says 172.21.15.64:36864) 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 -- 172.21.15.64:0/3360084040 learned_addr learned my addr 172.21.15.64:0/3360084040 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1525d9b700 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1526d9d700 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe25bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 -- 172.21.15.64:0/3360084040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 msgr2=0x55928fe25bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe25bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 -- 172.21.15.64:0/3360084040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 msgr2=0x55928fe34f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 -- 172.21.15.64:0/3360084040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55928fd3a510 con 0x55928fe244f0 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f1525d9b700 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f152659c700 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2e030 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f151009b380 tx=0x7f151009a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1e21c1ffff6bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f151d7fa700 1 -- 172.21.15.64:0/3360084040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15100a37e0 con 0x55928fe244f0 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f151d7fa700 1 -- 172.21.15.64:0/3360084040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1510081070 con 0x55928fe244f0 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.997+0000 7f151d7fa700 1 -- 172.21.15.64:0/3360084040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151009a350 con 0x55928fe244f0 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 -- 172.21.15.64:0/3360084040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 msgr2=0x55928fe2e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2e030 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f151009b380 tx=0x7f151009a8e0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 -- 172.21.15.64:0/3360084040 shutdown_connections 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe25810 0x55928fe25bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe244f0 0x55928fe2e030 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 --2- 172.21.15.64:0/3360084040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 -- 172.21.15.64:0/3360084040 >> 172.21.15.64:0/3360084040 conn(0x55928fd821c0 msgr2=0x55928fe2c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 -- 172.21.15.64:0/3360084040 shutdown_connections 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 -- 172.21.15.64:0/3360084040 wait complete. 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 -- start start 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.998+0000 7f1538db4b80 1 Processor -- start 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe244f0 0x55928fe31ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 0x55928fe34950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fe35040 con 0x55928fe25810 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fe351b0 con 0x55928fe24e50 2024-05-08T21:54:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55928fe8f660 con 0x55928fe244f0 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 10 client.0 _ll_register_callbacks cb 0x55928fd19ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 0x55928fe34950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 0x55928fe34950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36878/0 (socket says 172.21.15.64:36878) 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 -- 172.21.15.64:0/2866741147 learned_addr learned my addr 172.21.15.64:0/2866741147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f152659c700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe244f0 0x55928fe31ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1525d9b700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 -- 172.21.15.64:0/2866741147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe244f0 msgr2=0x55928fe31ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe244f0 0x55928fe31ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 -- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 msgr2=0x55928fe34410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 -- 172.21.15.64:0/2866741147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55928fba1b10 con 0x55928fe25810 2024-05-08T21:54:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1525d9b700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1526d9d700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 0x55928fe34950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f15200078c0 tx=0x7f1520007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb00a890049e4ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f150dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f150dffb700 1 -- 172.21.15.64:0/2866741147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200169d0 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15100a1050 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:14.999+0000 7f150dffb700 1 -- 172.21.15.64:0/2866741147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1520016b30 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f150dffb700 1 -- 172.21.15.64:0/2866741147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1520016cf0 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 10 client.15132 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55928fbf9c80 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f150dffb700 1 -- 172.21.15.64:0/2866741147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15100a1050 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f150dffb700 1 -- 172.21.15.64:0/2866741147 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1520014050 con 0x55928fe25810 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 20 client.15132 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe94380 0x55928fe90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 4 client.15132 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55928fd6ce70 con 0x55928fe94380 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.000+0000 7f152659c700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe94380 0x55928fe90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f150dffb700 10 client.15132 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f152659c700 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe94380 0x55928fe90d90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcbb65c4b342acda server_cookie=6feacf9c021e9223 in_seq=0 out_seq=0 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f150dffb700 1 -- 172.21.15.64:0/2866741147 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55928fd6ce70 con 0x55928fe94380 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f150dffb700 10 client.15132 handle_command_reply: tid=0 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 client.15132 shutdown 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 msgr2=0x55928fe34950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 0x55928fe34950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f15200078c0 tx=0x7f1520007bd0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 shutdown_connections 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55928fe94380 0x55928fe90d90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55928fe25810 0x55928fe34950 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f150dffb700 0 client.15132 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55928fe244f0 0x55928fe31ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 --2- 172.21.15.64:0/2866741147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55928fe24e50 0x55928fe34410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 >> 172.21.15.64:0/2866741147 conn(0x55928fd821c0 msgr2=0x55928fe25bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 shutdown_connections 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 1 -- 172.21.15.64:0/2866741147 wait complete. 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f1538db4b80 20 client.15132 trim_cache size 0 max 16384 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:15.010 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24506, 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 51.443748798999998, 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 57.567575986999998 2024-05-08T21:54:15.011 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15140a11b0 msgr2=0x7f15140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15140a11b0 0x7f15140a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1518005fd0 tx=0x7f1518005ee0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.001+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 msgr2=0x7f1528195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528195e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1510005910 tx=0x7f1510004730 comp rx=0 tx=0).stop 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 shutdown_connections 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15140a11b0 0x7f15140a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1528101970 0x7f152819d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15280ffe60 0x7f1528195e00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 --2- 172.21.15.64:0/1337368319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1528100fb0 0x7f1528196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:15.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 >> 172.21.15.64:0/1337368319 conn(0x7f15280fb3f0 msgr2=0x7f1528104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:15.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 shutdown_connections 2024-05-08T21:54:15.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:15.002+0000 7f152cbfa700 1 -- 172.21.15.64:0/1337368319 wait complete. 2024-05-08T21:54:15.015 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T21:54:15.016 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T21:54:15.025 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:54:15.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T21:54:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:15.539 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:15 smithi064 ceph-mon[42500]: pgmap v246: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:54:15.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:15 smithi160 ceph-mon[46962]: pgmap v246: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:54:15.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:15 smithi186 ceph-mon[46619]: pgmap v246: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:54:16.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 -- 172.21.15.64:0/4130567751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 msgr2=0x7fcdd8100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 --2- 172.21.15.64:0/4130567751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8100320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009990 tx=0x7fcdc8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 -- 172.21.15.64:0/4130567751 shutdown_connections 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 --2- 172.21.15.64:0/4130567751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd8101a20 0x7fcdd8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 --2- 172.21.15.64:0/4130567751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8100320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 --2- 172.21.15.64:0/4130567751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101060 0x7fcdd81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 -- 172.21.15.64:0/4130567751 >> 172.21.15.64:0/4130567751 conn(0x7fcdd80fb4c0 msgr2=0x7fcdd80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 -- 172.21.15.64:0/4130567751 shutdown_connections 2024-05-08T21:54:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.217+0000 7fcddd18c700 1 -- 172.21.15.64:0/4130567751 wait complete. 2024-05-08T21:54:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 Processor -- start 2024-05-08T21:54:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 -- start start 2024-05-08T21:54:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd8101060 0x7fcdd8196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101a20 0x7fcdd819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819d900 con 0x7fcdd80fff00 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819da40 con 0x7fcdd8101a20 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcddd18c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819dc00 con 0x7fcdd8101060 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcdd6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcdd6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36890/0 (socket says 172.21.15.64:36890) 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcdd6d9d700 1 -- 172.21.15.64:0/2835424649 learned_addr learned my addr 172.21.15.64:0/2835424649 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.218+0000 7fcdd659c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd8101060 0x7fcdd8196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd759e700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101a20 0x7fcdd819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd6d9d700 1 -- 172.21.15.64:0/2835424649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd8101060 msgr2=0x7fcdd8196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd6d9d700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd8101060 0x7fcdd8196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd6d9d700 1 -- 172.21.15.64:0/2835424649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101a20 msgr2=0x7fcdd819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd6d9d700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101a20 0x7fcdd819d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd6d9d700 1 -- 172.21.15.64:0/2835424649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80095f0 con 0x7fcdd80fff00 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd759e700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101a20 0x7fcdd819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdd6d9d700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8195e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8005910 tx=0x7fcdc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc801d070 con 0x7fcdd80fff00 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd80691e0 con 0x7fcdd80fff00 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdc8003f60 con 0x7fcdd80fff00 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8022cc0 con 0x7fcdd80fff00 2024-05-08T21:54:16.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.219+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd80696d0 con 0x7fcdd80fff00 2024-05-08T21:54:16.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.220+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd804ec60 con 0x7fcdd80fff00 2024-05-08T21:54:16.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.221+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcdc80040d0 con 0x7fcdd80fff00 2024-05-08T21:54:16.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.222+0000 7fcdbffff700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdc00a11b0 0x7fcdc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:16.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.222+0000 7fcdd659c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdc00a11b0 0x7fcdc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:16.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.222+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcdc80cdea0 con 0x7fcdd80fff00 2024-05-08T21:54:16.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.222+0000 7fcdd659c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdc00a11b0 0x7fcdc00a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00ce60 tx=0x7fcdcc00e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:16.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.226+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcdc8097730 con 0x7fcdd80fff00 2024-05-08T21:54:16.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.449+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdd80698f0 con 0x7fcdc00a11b0 2024-05-08T21:54:16.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.450+0000 7fcdbffff700 1 -- 172.21.15.64:0/2835424649 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fcdd80698f0 con 0x7fcdc00a11b0 2024-05-08T21:54:16.452 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2024-05-08T21:54:16.452 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2024-05-08T21:54:16.452 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2024-05-08T21:54:16.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.452+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdc00a11b0 msgr2=0x7fcdc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:16.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.452+0000 7fcddd18c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdc00a11b0 0x7fcdc00a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00ce60 tx=0x7fcdcc00e040 comp rx=0 tx=0).stop 2024-05-08T21:54:16.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.452+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 msgr2=0x7fcdd8195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.452+0000 7fcddd18c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8195e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8005910 tx=0x7fcdc8004730 comp rx=0 tx=0).stop 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 shutdown_connections 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdc00a11b0 0x7fcdc00a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd8101a20 0x7fcdd819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd80fff00 0x7fcdd8195e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 --2- 172.21.15.64:0/2835424649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd8101060 0x7fcdd8196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 >> 172.21.15.64:0/2835424649 conn(0x7fcdd80fb4c0 msgr2=0x7fcdd81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 shutdown_connections 2024-05-08T21:54:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:16.453+0000 7fcddd18c700 1 -- 172.21.15.64:0/2835424649 wait complete. 2024-05-08T21:54:16.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T21:54:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:17.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:17 smithi064 ceph-mon[42500]: pgmap v247: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:54:17.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:17 smithi064 ceph-mon[42500]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:17.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:17 smithi160 ceph-mon[46962]: pgmap v247: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:54:17.613 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:17 smithi160 ceph-mon[46962]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:17.644 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:17 smithi186 ceph-mon[46619]: pgmap v247: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:54:17.644 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:17 smithi186 ceph-mon[46619]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 -- 172.21.15.64:0/888701584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c1022a0 msgr2=0x7f1a0c102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/888701584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c1022a0 0x7f1a0c102700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009990 tx=0x7f19fc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 -- 172.21.15.64:0/888701584 shutdown_connections 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/888701584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0c0ff2c0 0x7f1a0c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/888701584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/888701584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c1022a0 0x7f1a0c102700 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 -- 172.21.15.64:0/888701584 >> 172.21.15.64:0/888701584 conn(0x7f1a0c0fac40 msgr2=0x7f1a0c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 -- 172.21.15.64:0/888701584 shutdown_connections 2024-05-08T21:54:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.066+0000 7f1a10bb3700 1 -- 172.21.15.64:0/888701584 wait complete. 2024-05-08T21:54:18.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 Processor -- start 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 -- start start 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a0c0ff2c0 0x7f1a0c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0c1022a0 0x7f1a0c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1a0ca0 con 0x7f1a0c102c40 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1a0de0 con 0x7f1a0c1022a0 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a10bb3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1a0f70 con 0x7f1a0c0ff2c0 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a0ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a0ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36916/0 (socket says 172.21.15.64:36916) 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a0ad9d700 1 -- 172.21.15.64:0/2862075002 learned_addr learned my addr 172.21.15.64:0/2862075002 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.067+0000 7f1a09d9b700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0c1022a0 0x7f1a0c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0a59c700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a0c0ff2c0 0x7f1a0c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0ad9d700 1 -- 172.21.15.64:0/2862075002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a0c0ff2c0 msgr2=0x7f1a0c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0ad9d700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a0c0ff2c0 0x7f1a0c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0ad9d700 1 -- 172.21.15.64:0/2862075002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0c1022a0 msgr2=0x7f1a0c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0ad9d700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0c1022a0 0x7f1a0c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0ad9d700 1 -- 172.21.15.64:0/2862075002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0095f0 con 0x7f1a0c102c40 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a0ad9d700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1a06d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400b9a0 tx=0x7f1a0400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400c930 con 0x7f1a0c102c40 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c1a1250 con 0x7f1a0c102c40 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a04014440 con 0x7f1a0c102c40 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040075d0 con 0x7f1a0c102c40 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.068+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c1a1750 con 0x7f1a0c102c40 2024-05-08T21:54:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.069+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c04ec60 con 0x7f1a0c102c40 2024-05-08T21:54:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.070+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1a040145b0 con 0x7f1a0c102c40 2024-05-08T21:54:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.071+0000 7f1a037fe700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19f80a11b0 0x7f19f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.071+0000 7f1a0a59c700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19f80a11b0 0x7f19f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.071+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a04006e40 con 0x7f1a0c102c40 2024-05-08T21:54:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.072+0000 7f1a0a59c700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19f80a11b0 0x7f19f80a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005950 tx=0x7f19f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:18.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.075+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1a04095bf0 con 0x7f1a0c102c40 2024-05-08T21:54:18.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.288+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a0c1017b0 con 0x7f19f80a11b0 2024-05-08T21:54:18.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.302+0000 7f1a037fe700 1 -- 172.21.15.64:0/2862075002 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1a0c1017b0 con 0x7f19f80a11b0 2024-05-08T21:54:18.303 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T21:54:18.303 INFO:teuthology.orchestra.run.smithi064.stdout:mds.a smithi064 running (62s) 57s ago 62s 13.4M - 18.2.2-1322-gd21e4f96 af23193ff691 68f5e4091226 2024-05-08T21:54:18.303 INFO:teuthology.orchestra.run.smithi064.stdout:mds.b smithi160 running (61s) 57s ago 60s 13.4M - 18.2.2-1322-gd21e4f96 af23193ff691 54d0438d1bf8 2024-05-08T21:54:18.303 INFO:teuthology.orchestra.run.smithi064.stdout:mds.c smithi186 running (70s) 57s ago 69s 13.4M - 18.2.2-1322-gd21e4f96 af23193ff691 83c94909c57b 2024-05-08T21:54:18.303 INFO:teuthology.orchestra.run.smithi064.stdout:mds.d smithi064 running (75s) 57s ago 74s 15.8M - 18.2.2-1322-gd21e4f96 af23193ff691 e9211df1fa61 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.e smithi160 running (73s) 57s ago 73s 13.4M - 18.2.2-1322-gd21e4f96 af23193ff691 20b940966dff 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.f smithi186 running (71s) 57s ago 71s 13.4M - 18.2.2-1322-gd21e4f96 af23193ff691 3b54f91c877e 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.g smithi064 running (64s) 57s ago 64s 14.2M - 18.2.2-1322-gd21e4f96 af23193ff691 d0e9ce30abc1 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.h smithi160 running (68s) 57s ago 67s 14.8M - 18.2.2-1322-gd21e4f96 af23193ff691 b50d12d54e9f 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.i smithi186 running (66s) 57s ago 66s 14.1M - 18.2.2-1322-gd21e4f96 af23193ff691 573eaf21e78a 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.j smithi064 running (76s) 57s ago 76s 16.5M - 18.2.2-1322-gd21e4f96 af23193ff691 3e49a7487444 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.k smithi160 running (78s) 57s ago 78s 13.2M - 18.2.2-1322-gd21e4f96 af23193ff691 21468ab5befb 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mds.l smithi186 running (80s) 57s ago 80s 14.1M - 18.2.2-1322-gd21e4f96 af23193ff691 b8393f83253f 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.x smithi064 *:9283,8765 running (6m) 57s ago 6m 523M - 18.2.2-1322-gd21e4f96 af23193ff691 31d83bc8920a 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.y smithi160 *:8443,8765 running (5m) 57s ago 5m 461M - 18.2.2-1322-gd21e4f96 af23193ff691 b738fb96d6bb 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.z smithi186 *:8443,8765 running (5m) 57s ago 5m 459M - 18.2.2-1322-gd21e4f96 af23193ff691 cbe67a60fa09 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mon.a smithi064 running (6m) 57s ago 6m 79.0M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 38c1f9ac2a9d 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mon.b smithi160 running (5m) 57s ago 5m 56.7M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 67d7947b225c 2024-05-08T21:54:18.304 INFO:teuthology.orchestra.run.smithi064.stdout:mon.c smithi186 running (5m) 57s ago 5m 54.2M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 c70294050cfa 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (4m) 57s ago 4m 66.7M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 f5b9a5002967 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi160 running (4m) 57s ago 4m 59.2M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 5f0fcbf700f1 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi186 running (4m) 57s ago 4m 53.9M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 4ce328bcdf5f 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi064 running (3m) 57s ago 3m 60.7M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 aadaa3cfe23c 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi160 running (3m) 57s ago 3m 47.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 26f7f08dfe70 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi186 running (3m) 57s ago 3m 60.7M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 a61d2cfa93d3 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 smithi064 running (3m) 57s ago 3m 50.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 a5d1b6300b0a 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 smithi160 running (2m) 57s ago 2m 51.5M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 2f404fc7f929 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 smithi186 running (2m) 57s ago 2m 48.4M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 22a18a3e8362 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 smithi064 running (2m) 57s ago 2m 45.4M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 db0d30e58453 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 smithi160 running (115s) 57s ago 115s 60.9M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 d2b4ebeee2ad 2024-05-08T21:54:18.305 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 smithi186 running (97s) 57s ago 97s 55.1M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 4cea3653d399 2024-05-08T21:54:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19f80a11b0 msgr2=0x7f19f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19f80a11b0 0x7f19f80a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005950 tx=0x7f19f40058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 msgr2=0x7f1a0c1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1a06d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400b9a0 tx=0x7f1a0400bcb0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 shutdown_connections 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f19f80a11b0 0x7f19f80a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c102c40 0x7f1a0c1a06d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a0c0ff2c0 0x7f1a0c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 --2- 172.21.15.64:0/2862075002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0c1022a0 0x7f1a0c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 >> 172.21.15.64:0/2862075002 conn(0x7f1a0c0fac40 msgr2=0x7f1a0c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 shutdown_connections 2024-05-08T21:54:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:18.304+0000 7f1a10bb3700 1 -- 172.21.15.64:0/2862075002 wait complete. 2024-05-08T21:54:18.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T21:54:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:19.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:19 smithi064 ceph-mon[42500]: pgmap v248: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T21:54:19.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:19 smithi064 ceph-mon[42500]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:19.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:19 smithi160 ceph-mon[46962]: pgmap v248: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T21:54:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:19 smithi160 ceph-mon[46962]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:19 smithi186 ceph-mon[46619]: pgmap v248: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T21:54:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:19 smithi186 ceph-mon[46619]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 -- 172.21.15.64:0/1594286400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2890101a90 msgr2=0x7f2890103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 --2- 172.21.15.64:0/1594286400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2890101a90 0x7f2890103e80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2880009990 tx=0x7f2880009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 -- 172.21.15.64:0/1594286400 shutdown_connections 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 --2- 172.21.15.64:0/1594286400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2890106cf0 0x7f2890109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 --2- 172.21.15.64:0/1594286400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2890101a90 0x7f2890103e80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 --2- 172.21.15.64:0/1594286400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28901043c0 0x7f28901067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.940+0000 7f2895c12700 1 -- 172.21.15.64:0/1594286400 >> 172.21.15.64:0/1594286400 conn(0x7f28900fb4a0 msgr2=0x7f28900fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.941+0000 7f2895c12700 1 -- 172.21.15.64:0/1594286400 shutdown_connections 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.941+0000 7f2895c12700 1 -- 172.21.15.64:0/1594286400 wait complete. 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.941+0000 7f2895c12700 1 Processor -- start 2024-05-08T21:54:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.941+0000 7f2895c12700 1 -- start start 2024-05-08T21:54:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2890101a90 0x7f28901013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 0x7f28900ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2890106cf0 0x7f28900fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890100720 con 0x7f28901043c0 2024-05-08T21:54:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890100890 con 0x7f2890101a90 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28901a1020 con 0x7f2890106cf0 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 0x7f28900ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 0x7f28900ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36940/0 (socket says 172.21.15.64:36940) 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 -- 172.21.15.64:0/2930638966 learned_addr learned my addr 172.21.15.64:0/2930638966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288f7fe700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2890101a90 0x7f28901013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288ffff700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2890106cf0 0x7f28900fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 -- 172.21.15.64:0/2930638966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2890106cf0 msgr2=0x7f28900fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2890106cf0 0x7f28900fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 -- 172.21.15.64:0/2930638966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2890101a90 msgr2=0x7f28901013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2890101a90 0x7f28901013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 -- 172.21.15.64:0/2930638966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28800095f0 con 0x7f28901043c0 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288ffff700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2890106cf0 0x7f28900fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288effd700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 0x7f28900ffa40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f288400b530 tx=0x7f288400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884011830 con 0x7f28901043c0 2024-05-08T21:54:19.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2884011e50 con 0x7f28901043c0 2024-05-08T21:54:19.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400f2e0 con 0x7f28901043c0 2024-05-08T21:54:19.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28901a12b0 con 0x7f28901043c0 2024-05-08T21:54:19.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.942+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28901a17b0 con 0x7f28901043c0 2024-05-08T21:54:19.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.943+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28900fd020 con 0x7f28901043c0 2024-05-08T21:54:19.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.945+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2884010380 con 0x7f28901043c0 2024-05-08T21:54:19.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.945+0000 7f288cff9700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f28780a1230 0x7f28780a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:19.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.945+0000 7f288f7fe700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f28780a1230 0x7f28780a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:19.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.945+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f28840cc830 con 0x7f28901043c0 2024-05-08T21:54:19.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.946+0000 7f288f7fe700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f28780a1230 0x7f28780a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f288000b5c0 tx=0x7f2880003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:19.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:19.949+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2884096120 con 0x7f28901043c0 2024-05-08T21:54:20.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.163+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f28900613e0 con 0x7f28780a1230 2024-05-08T21:54:20.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.168+0000 7f288cff9700 1 -- 172.21.15.64:0/2930638966 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f28900613e0 con 0x7f28780a1230 2024-05-08T21:54:20.169 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T21:54:20.170 INFO:teuthology.orchestra.run.smithi064.stdout:mds.all 12/12 59s ago 84s smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-08T21:54:20.170 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 3/3 59s ago 5m smithi064=x;smithi160=y;smithi186=z;count:3 2024-05-08T21:54:20.170 INFO:teuthology.orchestra.run.smithi064.stdout:mon 3/3 59s ago 5m smithi064:172.21.15.64=a;smithi160:172.21.15.160=b;smithi186:172.21.15.186=c;count:3 2024-05-08T21:54:20.170 INFO:teuthology.orchestra.run.smithi064.stdout:osd 12 59s ago - 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f28780a1230 msgr2=0x7f28780a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f28780a1230 0x7f28780a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f288000b5c0 tx=0x7f2880003790 comp rx=0 tx=0).stop 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 msgr2=0x7f28900ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 0x7f28900ffa40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f288400b530 tx=0x7f288400b840 comp rx=0 tx=0).stop 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 shutdown_connections 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f28780a1230 0x7f28780a36f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2890101a90 0x7f28901013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2890106cf0 0x7f28900fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 --2- 172.21.15.64:0/2930638966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28901043c0 0x7f28900ffa40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 >> 172.21.15.64:0/2930638966 conn(0x7f28900fb4a0 msgr2=0x7f2890103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:20.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 shutdown_connections 2024-05-08T21:54:20.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:20.170+0000 7f2895c12700 1 -- 172.21.15.64:0/2930638966 wait complete. 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-08T21:54:20.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:20 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-08T21:54:20.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:20 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-08T21:54:20.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-08T21:54:20.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:20 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-08T21:54:20.804 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: pgmap v249: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T21:54:21.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:21 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:54:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: pgmap v249: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:54:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-08T21:54:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-08T21:54:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-08T21:54:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-08T21:54:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-05-08T21:54:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-08T21:54:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-08T21:54:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-08T21:54:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T21:54:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:21 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:54:21.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: pgmap v249: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:54:21.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:21.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-08T21:54:21.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-08T21:54:21.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-08T21:54:21.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-08T21:54:21.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-05-08T21:54:21.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-08T21:54:21.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-08T21:54:21.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-08T21:54:21.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T21:54:21.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:21 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 -- 172.21.15.64:0/1333697021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c073a00 msgr2=0x7fc07c073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 --2- 172.21.15.64:0/1333697021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c073a00 0x7fc07c073e80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 -- 172.21.15.64:0/1333697021 shutdown_connections 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 --2- 172.21.15.64:0/1333697021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c108ed0 0x7fc07c10b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 --2- 172.21.15.64:0/1333697021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c074fd0 0x7fc07c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 --2- 172.21.15.64:0/1333697021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c073a00 0x7fc07c073e80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 -- 172.21.15.64:0/1333697021 >> 172.21.15.64:0/1333697021 conn(0x7fc07c0fc5a0 msgr2=0x7fc07c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 -- 172.21.15.64:0/1333697021 shutdown_connections 2024-05-08T21:54:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.871+0000 7fc0834e0700 1 -- 172.21.15.64:0/1333697021 wait complete. 2024-05-08T21:54:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 Processor -- start 2024-05-08T21:54:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 -- start start 2024-05-08T21:54:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c073a00 0x7fc07c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c074fd0 0x7fc07c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 0x7fc07c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a0cd0 con 0x7fc07c108ed0 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a0e10 con 0x7fc07c074fd0 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc0834e0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a0fa0 con 0x7fc07c073a00 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc081a7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 0x7fc07c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc08127c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c073a00 0x7fc07c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc080a7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c074fd0 0x7fc07c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc081a7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 0x7fc07c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36958/0 (socket says 172.21.15.64:36958) 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.872+0000 7fc081a7d700 1 -- 172.21.15.64:0/993883567 learned_addr learned my addr 172.21.15.64:0/993883567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc081a7d700 1 -- 172.21.15.64:0/993883567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c073a00 msgr2=0x7fc07c199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc081a7d700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c073a00 0x7fc07c199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc081a7d700 1 -- 172.21.15.64:0/993883567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c074fd0 msgr2=0x7fc07c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc081a7d700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c074fd0 0x7fc07c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc081a7d700 1 -- 172.21.15.64:0/993883567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0095f0 con 0x7fc07c108ed0 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc080a7b700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c074fd0 0x7fc07c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc081a7d700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 0x7fc07c1a0700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc07400ef60 tx=0x7fc07400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074018070 con 0x7fc07c108ed0 2024-05-08T21:54:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07400f040 con 0x7fc07c108ed0 2024-05-08T21:54:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07400ccc0 con 0x7fc07c108ed0 2024-05-08T21:54:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c1a1220 con 0x7fc07c108ed0 2024-05-08T21:54:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.873+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c1a16d0 con 0x7fc07c108ed0 2024-05-08T21:54:21.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.875+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c107c90 con 0x7fc07c108ed0 2024-05-08T21:54:21.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.875+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc074003a00 con 0x7fc07c108ed0 2024-05-08T21:54:21.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.876+0000 7fc0727fc700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc0680a1230 0x7fc0680a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:21.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.876+0000 7fc08127c700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc0680a1230 0x7fc0680a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:21.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.876+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0740cc4d0 con 0x7fc07c108ed0 2024-05-08T21:54:21.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.876+0000 7fc08127c700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc0680a1230 0x7fc0680a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0753b0 tx=0x7fc078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:21.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:21.880+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc074095b40 con 0x7fc07c108ed0 2024-05-08T21:54:22.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.112+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc07c0613e0 con 0x7fc0680a1230 2024-05-08T21:54:22.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.117+0000 7fc0727fc700 1 -- 172.21.15.64:0/993883567 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fc07c0613e0 con 0x7fc0680a1230 2024-05-08T21:54:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2024-05-08T21:54:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2024-05-08T21:54:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:smithi160 172.21.15.160 2024-05-08T21:54:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:smithi186 172.21.15.186 2024-05-08T21:54:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:3 hosts in cluster 2024-05-08T21:54:22.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc0680a1230 msgr2=0x7fc0680a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:22.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc0680a1230 0x7fc0680a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0753b0 tx=0x7fc078009260 comp rx=0 tx=0).stop 2024-05-08T21:54:22.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 msgr2=0x7fc07c1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:22.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 0x7fc07c1a0700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc07400ef60 tx=0x7fc07400ef90 comp rx=0 tx=0).stop 2024-05-08T21:54:22.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 shutdown_connections 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc0680a1230 0x7fc0680a36f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c108ed0 0x7fc07c1a0700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c073a00 0x7fc07c199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 --2- 172.21.15.64:0/993883567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc07c074fd0 0x7fc07c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 >> 172.21.15.64:0/993883567 conn(0x7fc07c0fc5a0 msgr2=0x7fc07c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 shutdown_connections 2024-05-08T21:54:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:22.119+0000 7fc0834e0700 1 -- 172.21.15.64:0/993883567 wait complete. 2024-05-08T21:54:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:22 smithi064 ceph-mon[42500]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T21:54:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:22 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T21:54:22.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:22 smithi160 ceph-mon[46962]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T21:54:22.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:22 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:22 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:22 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:22 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:22 smithi186 ceph-mon[46619]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T21:54:22.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:22 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:22 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:22 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:22 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:23.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: pgmap v252: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:54:23.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:23.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:23.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:23.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:54:23.693 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:54:23.693 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 -- 172.21.15.64:0/3727049603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 msgr2=0x7f7be01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 --2- 172.21.15.64:0/3727049603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be01014e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009990 tx=0x7f7bd0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 -- 172.21.15.64:0/3727049603 shutdown_connections 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 --2- 172.21.15.64:0/3727049603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 0x7f7be0108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 --2- 172.21.15.64:0/3727049603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 0x7f7be0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 --2- 172.21.15.64:0/3727049603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be01014e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 -- 172.21.15.64:0/3727049603 >> 172.21.15.64:0/3727049603 conn(0x7f7be00fb4a0 msgr2=0x7f7be00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 -- 172.21.15.64:0/3727049603 shutdown_connections 2024-05-08T21:54:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.726+0000 7f7be550e700 1 -- 172.21.15.64:0/3727049603 wait complete. 2024-05-08T21:54:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 Processor -- start 2024-05-08T21:54:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 -- start start 2024-05-08T21:54:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 0x7f7be0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 0x7f7be019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019e9c0 con 0x7f7be0101060 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019eb30 con 0x7f7be0101a20 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7be550e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019ecf0 con 0x7f7be00fff00 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7bde7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7bde7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38954/0 (socket says 172.21.15.64:38954) 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.727+0000 7f7bde7fc700 1 -- 172.21.15.64:0/3913678193 learned_addr learned my addr 172.21.15.64:0/3913678193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bdeffd700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 0x7f7be0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bdf7fe700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 0x7f7be019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bde7fc700 1 -- 172.21.15.64:0/3913678193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 msgr2=0x7f7be0069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bde7fc700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 0x7f7be0069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bde7fc700 1 -- 172.21.15.64:0/3913678193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 msgr2=0x7f7be019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bde7fc700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 0x7f7be019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bde7fc700 1 -- 172.21.15.64:0/3913678193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd00095f0 con 0x7f7be0101060 2024-05-08T21:54:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bdeffd700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 0x7f7be0069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bdf7fe700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 0x7f7be019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bde7fc700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be019bc50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000b5c0 tx=0x7f7bd0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd001d070 con 0x7f7be0101060 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be019ef70 con 0x7f7be0101060 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd0004060 con 0x7f7be0101060 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0022e10 con 0x7f7be0101060 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.728+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be019f320 con 0x7f7be0101060 2024-05-08T21:54:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.729+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be01073c0 con 0x7f7be0101060 2024-05-08T21:54:23.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.730+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7bd0022570 con 0x7f7be0101060 2024-05-08T21:54:23.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.731+0000 7f7bc7fff700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7bc80a11b0 0x7f7bc80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:23.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.731+0000 7f7bdeffd700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7bc80a11b0 0x7f7bc80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:23.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.731+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7bd00ce2b0 con 0x7f7be0101060 2024-05-08T21:54:23.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.731+0000 7f7bdeffd700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7bc80a11b0 0x7f7bc80a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4005950 tx=0x7f7bd4005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:23.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.734+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7bd0097a10 con 0x7f7be0101060 2024-05-08T21:54:23.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.947+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 --> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7be002ceb0 con 0x7f7bc80a11b0 2024-05-08T21:54:23.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.949+0000 7f7bc7fff700 1 -- 172.21.15.64:0/3913678193 <== mgr.14152 v2:172.21.15.64:6800/3946872331 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f7be002ceb0 con 0x7f7bc80a11b0 2024-05-08T21:54:23.950 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T21:54:23.951 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T21:54:23.951 INFO:teuthology.orchestra.run.smithi064.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 116s ago Has a FileSystem, LVM detected 2024-05-08T21:54:23.951 INFO:teuthology.orchestra.run.smithi064.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 98s ago Has a FileSystem, LVM detected 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7bc80a11b0 msgr2=0x7f7bc80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7bc80a11b0 0x7f7bc80a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4005950 tx=0x7f7bd4005f50 comp rx=0 tx=0).stop 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 msgr2=0x7f7be019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be019bc50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000b5c0 tx=0x7f7bd0005e80 comp rx=0 tx=0).stop 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 shutdown_connections 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7bc80a11b0 0x7f7bc80a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7be0101a20 0x7f7be019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be00fff00 0x7f7be0069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 --2- 172.21.15.64:0/3913678193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be0101060 0x7f7be019bc50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 >> 172.21.15.64:0/3913678193 conn(0x7f7be00fb4a0 msgr2=0x7f7be0104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 shutdown_connections 2024-05-08T21:54:23.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:23.951+0000 7f7be550e700 1 -- 172.21.15.64:0/3913678193 wait complete. 2024-05-08T21:54:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: pgmap v252: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:54:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:54:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:54:24.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: pgmap v252: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:54:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:24.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:54:24.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:54:24.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:54:24.332 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T21:54:24.341 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:54:24.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T21:54:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:24.696 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:24 smithi064 ceph-mon[42500]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:25.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:24 smithi160 ceph-mon[46962]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:25.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:24 smithi186 ceph-mon[46619]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:54:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 -- 172.21.15.64:0/996925178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24102ba0 msgr2=0x7efe24109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 --2- 172.21.15.64:0/996925178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24102ba0 0x7efe24109450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b0d0 tx=0x7efe1c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 -- 172.21.15.64:0/996925178 shutdown_connections 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 --2- 172.21.15.64:0/996925178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24102ba0 0x7efe24109450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 --2- 172.21.15.64:0/996925178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24101000 0x7efe24101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 --2- 172.21.15.64:0/996925178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24102200 0x7efe24102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 -- 172.21.15.64:0/996925178 >> 172.21.15.64:0/996925178 conn(0x7efe240fc5c0 msgr2=0x7efe240fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 -- 172.21.15.64:0/996925178 shutdown_connections 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.557+0000 7efe2974a700 1 -- 172.21.15.64:0/996925178 wait complete. 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 Processor -- start 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 -- start start 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 0x7efe24196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24102200 0x7efe241974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24102ba0 0x7efe2419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419eae0 con 0x7efe24101000 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419ec20 con 0x7efe24102200 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe2974a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419edb0 con 0x7efe24102ba0 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe22ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 0x7efe24196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe22ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 0x7efe24196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38972/0 (socket says 172.21.15.64:38972) 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe22ffd700 1 -- 172.21.15.64:0/3803145503 learned_addr learned my addr 172.21.15.64:0/3803145503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.558+0000 7efe237fe700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24102ba0 0x7efe2419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe227fc700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24102200 0x7efe241974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe22ffd700 1 -- 172.21.15.64:0/3803145503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24102ba0 msgr2=0x7efe2419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe22ffd700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24102ba0 0x7efe2419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe22ffd700 1 -- 172.21.15.64:0/3803145503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24102200 msgr2=0x7efe241974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe22ffd700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24102200 0x7efe241974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe22ffd700 1 -- 172.21.15.64:0/3803145503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c009d00 con 0x7efe24101000 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe22ffd700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 0x7efe24196fa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d9b0 tx=0x7efe1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14004020 con 0x7efe24101000 2024-05-08T21:54:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14009730 con 0x7efe24101000 2024-05-08T21:54:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe24073230 con 0x7efe24101000 2024-05-08T21:54:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14003a30 con 0x7efe24101000 2024-05-08T21:54:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.559+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe24073730 con 0x7efe24101000 2024-05-08T21:54:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.560+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe24108440 con 0x7efe24101000 2024-05-08T21:54:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.561+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efe140098a0 con 0x7efe24101000 2024-05-08T21:54:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.562+0000 7efe0bfff700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe0c0a11b0 0x7efe0c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.562+0000 7efe227fc700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe0c0a11b0 0x7efe0c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.562+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe140ccab0 con 0x7efe24101000 2024-05-08T21:54:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.562+0000 7efe227fc700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe0c0a11b0 0x7efe0c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe18005950 tx=0x7efe180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.565+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efe140960f0 con 0x7efe24101000 2024-05-08T21:54:25.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.814+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe2402cda0 con 0x7efe24101000 2024-05-08T21:54:25.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.814+0000 7efe0bfff700 1 -- 172.21.15.64:0/3803145503 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2929 (secure 0 0 0) 0x7efe14017b90 con 0x7efe24101000 2024-05-08T21:54:25.816 INFO:teuthology.orchestra.run.smithi064.stdout:e27 2024-05-08T21:54:25.816 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:54:25.816 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:54:25.816 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 27 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-05-08T21:53:23.555551+0000 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-05-08T21:53:29.959964+0000 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 300 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-05-08T21:54:25.817 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 5 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:in 0,1,2,3,4 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24506,1=24484,2=24476,3=24427,4=24422} 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-05-08T21:54:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b{0:24506} state up:active seq 3 addr [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.186:6836/3888486173,v1:172.21.15.186:6837/3888486173] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.i{1:24484} state up:active seq 5 addr [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.f{1:24454} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/4069999836,v1:172.21.15.186:6835/4069999836] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.h{2:24476} state up:active seq 6 addr [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.e{2:24452} state up:standby-replay seq 1 addr [v2:172.21.15.160:6834/770617410,v1:172.21.15.160:6835/770617410] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.l{3:24427} state up:active seq 9 addr [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a{3:14637} state up:standby-replay seq 1 addr [v2:172.21.15.64:6840/1669365277,v1:172.21.15.64:6841/1669365277] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.k{4:24422} state up:active seq 9 addr [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.g{4:14625} state up:standby-replay seq 1 addr [v2:172.21.15.64:6838/3760541428,v1:172.21.15.64:6839/3760541428] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-05-08T21:54:25.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:25.820 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.64:6834/1013310414,v1:172.21.15.64:6835/1013310414] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.820 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.64:6836/2682658639,v1:172.21.15.64:6837/2682658639] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:54:25.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.816+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe0c0a11b0 msgr2=0x7efe0c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe0c0a11b0 0x7efe0c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe18005950 tx=0x7efe180058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 msgr2=0x7efe24196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 0x7efe24196fa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d9b0 tx=0x7efe1400dcc0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 shutdown_connections 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe0c0a11b0 0x7efe0c0a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24102ba0 0x7efe2419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe24101000 0x7efe24196fa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 --2- 172.21.15.64:0/3803145503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24102200 0x7efe241974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 >> 172.21.15.64:0/3803145503 conn(0x7efe240fc5c0 msgr2=0x7efe24105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 shutdown_connections 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:25.817+0000 7efe2974a700 1 -- 172.21.15.64:0/3803145503 wait complete. 2024-05-08T21:54:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:54:25.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:25 smithi064 ceph-mon[42500]: pgmap v253: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:25 smithi160 ceph-mon[46962]: pgmap v253: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:25 smithi186 ceph-mon[46619]: pgmap v253: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:26.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T21:54:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:26.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:26 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3803145503' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T21:54:27.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:26 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3803145503' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T21:54:27.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:26 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3803145503' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T21:54:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 -- 172.21.15.64:0/3085197081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 msgr2=0x7f135c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 --2- 172.21.15.64:0/3085197081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c106810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1348009990 tx=0x7f1348009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 -- 172.21.15.64:0/3085197081 shutdown_connections 2024-05-08T21:54:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 --2- 172.21.15.64:0/3085197081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135c101af0 0x7f135c103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 --2- 172.21.15.64:0/3085197081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f135c106d50 0x7f135c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 --2- 172.21.15.64:0/3085197081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c106810 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 -- 172.21.15.64:0/3085197081 >> 172.21.15.64:0/3085197081 conn(0x7f135c0fb4a0 msgr2=0x7f135c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 -- 172.21.15.64:0/3085197081 shutdown_connections 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.373+0000 7f1362ddb700 1 -- 172.21.15.64:0/3085197081 wait complete. 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 Processor -- start 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 -- start start 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f135c101af0 0x7f135c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c0ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135c106d50 0x7f135c0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c100780 con 0x7f135c104420 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c1008f0 con 0x7f135c106d50 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1362ddb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c1a0ff0 con 0x7f135c101af0 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c0ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c0ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38978/0 (socket says 172.21.15.64:38978) 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 -- 172.21.15.64:0/2389123849 learned_addr learned my addr 172.21.15.64:0/2389123849 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1360b77700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f135c101af0 0x7f135c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f1361378700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135c106d50 0x7f135c0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 -- 172.21.15.64:0/2389123849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f135c101af0 msgr2=0x7f135c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f135c101af0 0x7f135c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 -- 172.21.15.64:0/2389123849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135c106d50 msgr2=0x7f135c0fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135c106d50 0x7f135c0fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.374+0000 7f135bfff700 1 -- 172.21.15.64:0/2389123849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480095f0 con 0x7f135c104420 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.375+0000 7f135bfff700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c0ffaa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13480095c0 tx=0x7f134800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.375+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134801d070 con 0x7f135c104420 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.375+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1348005030 con 0x7f135c104420 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.375+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c1a1220 con 0x7f135c104420 2024-05-08T21:54:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.375+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348004a50 con 0x7f135c104420 2024-05-08T21:54:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.375+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c1a1630 con 0x7f135c104420 2024-05-08T21:54:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.376+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135c0fd020 con 0x7f135c104420 2024-05-08T21:54:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.377+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1348005450 con 0x7f135c104420 2024-05-08T21:54:27.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.378+0000 7f1359ffb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f134c0a11b0 0x7f134c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:27.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.378+0000 7f1360b77700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f134c0a11b0 0x7f134c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:27.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.378+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f13480cdcc0 con 0x7f135c104420 2024-05-08T21:54:27.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.378+0000 7f1360b77700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f134c0a11b0 0x7f134c0a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1350007c60 tx=0x7f13500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.381+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1348097390 con 0x7f135c104420 2024-05-08T21:54:27.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.587+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f135c04ec60 con 0x7f135c104420 2024-05-08T21:54:27.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.588+0000 7f1359ffb700 1 -- 172.21.15.64:0/2389123849 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) v1 ==== 56+0+5281 (secure 0 0 0) 0x7f134809a9b0 con 0x7f135c104420 2024-05-08T21:54:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 81 2024-05-08T21:54:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T21:54:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-05-08T21:47:28.948985+0000 2024-05-08T21:54:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-05-08T21:54:21.242448+0000 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:crush_version 26 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:full_ratio 0.95 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:backfillfull_ratio 0.9 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:nearfull_ratio 0.85 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:require_min_compat_client luminous 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:min_compat_client luminous 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:require_osd_release reef 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:stretch_mode_enabled false 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-08T21:54:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:max_osd 12 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6802/4148876663,v1:172.21.15.64:6803/4148876663] [v2:172.21.15.64:6804/4148876663,v1:172.21.15.64:6805/4148876663] exists,up 1bb0a350-33bc-4f0d-a340-96bb54726c36 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6800/255573716,v1:172.21.15.160:6801/255573716] [v2:172.21.15.160:6802/255573716,v1:172.21.15.160:6803/255573716] exists,up 0fb69147-6892-4a82-8c67-94919896a9ac 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/348681043,v1:172.21.15.186:6801/348681043] [v2:172.21.15.186:6802/348681043,v1:172.21.15.186:6803/348681043] exists,up 65e106b3-aa74-4293-98f6-09717bbfca26 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6810/3390967060,v1:172.21.15.64:6811/3390967060] [v2:172.21.15.64:6812/3390967060,v1:172.21.15.64:6813/3390967060] exists,up 6a840d5a-a687-4cf2-a946-7a2f46bdfd26 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6808/1821714303,v1:172.21.15.160:6809/1821714303] [v2:172.21.15.160:6810/1821714303,v1:172.21.15.160:6811/1821714303] exists,up e00f8790-f664-4e52-b320-ab5e454f0235 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 up in weight 1 up_from 36 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/1348911288,v1:172.21.15.186:6809/1348911288] [v2:172.21.15.186:6810/1348911288,v1:172.21.15.186:6811/1348911288] exists,up 7e44e11d-06e2-40ef-9bf4-43bfd19c99da 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 up in weight 1 up_from 43 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6818/3400846142,v1:172.21.15.64:6819/3400846142] [v2:172.21.15.64:6820/3400846142,v1:172.21.15.64:6821/3400846142] exists,up e46cfa92-0895-463b-92e8-a8ab73a0d1e6 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 up in weight 1 up_from 48 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6816/3316502570,v1:172.21.15.160:6817/3316502570] [v2:172.21.15.160:6818/3316502570,v1:172.21.15.160:6819/3316502570] exists,up 9dc1fffb-253b-47fb-99de-85b4c9cfa6e3 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 up in weight 1 up_from 55 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/4072780638,v1:172.21.15.186:6817/4072780638] [v2:172.21.15.186:6818/4072780638,v1:172.21.15.186:6819/4072780638] exists,up 316f22c0-33f7-44c2-ae4f-0ed122d52d5f 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 up in weight 1 up_from 62 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6826/200787259,v1:172.21.15.64:6827/200787259] [v2:172.21.15.64:6828/200787259,v1:172.21.15.64:6829/200787259] exists,up 0bbe8d1f-ece5-435b-89db-1abeefdb4320 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 up in weight 1 up_from 67 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6824/1117750760,v1:172.21.15.160:6825/1117750760] [v2:172.21.15.160:6826/1117750760,v1:172.21.15.160:6827/1117750760] exists,up 63c6380e-f30e-4139-a9a3-6250bb4c4358 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 up in weight 1 up_from 74 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/2726552436,v1:172.21.15.186:6825/2726552436] [v2:172.21.15.186:6826/2726552436,v1:172.21.15.186:6827/2726552436] exists,up a05f3067-983c-4656-9914-05ab90f5bd9d 2024-05-08T21:54:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.6 [8,11] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.8 [2,11] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.16 [2,4] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.0 [4,8] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.7 [3,9] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1b [4,6] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.20 [5,9] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.24 [4,9] 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1096250282 expires 2024-05-09T21:48:20.034470+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/865555054 expires 2024-05-09T21:48:20.034470+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/865555054 expires 2024-05-09T21:48:20.034470+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/2750019664 expires 2024-05-09T21:48:20.034470+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3310656394 expires 2024-05-09T21:47:59.744672+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/1406677494 expires 2024-05-09T21:47:59.744672+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1642224217 expires 2024-05-09T21:47:59.744672+0000 2024-05-08T21:54:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3843761611 expires 2024-05-09T21:47:59.744672+0000 2024-05-08T21:54:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/1406677494 expires 2024-05-09T21:47:59.744672+0000 2024-05-08T21:54:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3369547289 expires 2024-05-09T21:48:20.034470+0000 2024-05-08T21:54:27.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f134c0a11b0 msgr2=0x7f134c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:27.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f134c0a11b0 0x7f134c0a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1350007c60 tx=0x7f13500073d0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 msgr2=0x7f135c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:27.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c0ffaa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13480095c0 tx=0x7f134800f690 comp rx=0 tx=0).stop 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 shutdown_connections 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f134c0a11b0 0x7f134c0a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135c106d50 0x7f135c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f135c101af0 0x7f135c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 --2- 172.21.15.64:0/2389123849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135c104420 0x7f135c0ffaa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 >> 172.21.15.64:0/2389123849 conn(0x7f135c0fb4a0 msgr2=0x7f135c103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 shutdown_connections 2024-05-08T21:54:27.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:27.590+0000 7f1362ddb700 1 -- 172.21.15.64:0/2389123849 wait complete. 2024-05-08T21:54:27.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:27 smithi064 ceph-mon[42500]: pgmap v254: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:27.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2389123849' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T21:54:27.966 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T21:54:27.977 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:54:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:27 smithi160 ceph-mon[46962]: pgmap v254: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:28.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2389123849' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T21:54:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 -- 172.21.15.64:0/3674663803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981018f0 msgr2=0x7fa6981081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/3674663803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981018f0 0x7fa6981081a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b0d0 tx=0x7fa69000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 -- 172.21.15.64:0/3674663803 shutdown_connections 2024-05-08T21:54:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/3674663803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981018f0 0x7fa6981081a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/3674663803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6980ffd60 0x7fa698100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/3674663803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa698100f50 0x7fa6981013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 -- 172.21.15.64:0/3674663803 >> 172.21.15.64:0/3674663803 conn(0x7fa6980fb2f0 msgr2=0x7fa6980fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 -- 172.21.15.64:0/3674663803 shutdown_connections 2024-05-08T21:54:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.106+0000 7fa69dc8d700 1 -- 172.21.15.64:0/3674663803 wait complete. 2024-05-08T21:54:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 Processor -- start 2024-05-08T21:54:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 -- start start 2024-05-08T21:54:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ffd60 0x7fa698068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 0x7fa69819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6981018f0 0x7fa69819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69819ea50 con 0x7fa698100f50 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69819ebc0 con 0x7fa6980ffd60 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.107+0000 7fa69dc8d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69819ed80 con 0x7fa6981018f0 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 0x7fa69819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 0x7fa69819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39006/0 (socket says 172.21.15.64:39006) 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 -- 172.21.15.64:0/1721894956 learned_addr learned my addr 172.21.15.64:0/1721894956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa6977fe700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ffd60 0x7fa698068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 -- 172.21.15.64:0/1721894956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6981018f0 msgr2=0x7fa69819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6981018f0 0x7fa69819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 -- 172.21.15.64:0/1721894956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ffd60 msgr2=0x7fa698068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ffd60 0x7fa698068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 -- 172.21.15.64:0/1721894956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690009d00 con 0x7fa698100f50 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa696ffd700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 0x7fa69819bcc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b530 tx=0x7fa68000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680011830 con 0x7fa698100f50 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69806aa20 con 0x7fa698100f50 2024-05-08T21:54:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa680011e50 con 0x7fa698100f50 2024-05-08T21:54:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.108+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68000f2e0 con 0x7fa698100f50 2024-05-08T21:54:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.109+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69806afb0 con 0x7fa698100f50 2024-05-08T21:54:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.109+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa698107110 con 0x7fa698100f50 2024-05-08T21:54:28.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.111+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa680010380 con 0x7fa698100f50 2024-05-08T21:54:28.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.111+0000 7fa694ff9700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6840a11b0 0x7fa6840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.111+0000 7fa6977fe700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6840a11b0 0x7fa6840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.111+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa6800cca30 con 0x7fa698100f50 2024-05-08T21:54:28.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.112+0000 7fa6977fe700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6840a11b0 0x7fa6840a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa698100db0 tx=0x7fa68800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.115+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa680096180 con 0x7fa698100f50 2024-05-08T21:54:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:27 smithi186 ceph-mon[46619]: pgmap v254: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2389123849' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T21:54:28.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.325+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa69810b640 con 0x7fa698100f50 2024-05-08T21:54:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.327+0000 7fa694ff9700 1 -- 172.21.15.64:0/1721894956 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22495 (secure 0 0 0) 0x7fa680015090 con 0x7fa698100f50 2024-05-08T21:54:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:28.328 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":81,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:54:21.242448+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:54:28.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6840a11b0 msgr2=0x7fa6840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6840a11b0 0x7fa6840a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa698100db0 tx=0x7fa68800b410 comp rx=0 tx=0).stop 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 msgr2=0x7fa69819bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 0x7fa69819bcc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b530 tx=0x7fa68000b840 comp rx=0 tx=0).stop 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 shutdown_connections 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6840a11b0 0x7fa6840a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ffd60 0x7fa698068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6981018f0 0x7fa69819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 --2- 172.21.15.64:0/1721894956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa698100f50 0x7fa69819bcc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 >> 172.21.15.64:0/1721894956 conn(0x7fa6980fb2f0 msgr2=0x7fa698104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 shutdown_connections 2024-05-08T21:54:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.329+0000 7fa69dc8d700 1 -- 172.21.15.64:0/1721894956 wait complete. 2024-05-08T21:54:28.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:53:21.578075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:53:22.933075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:54:28.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 -- 172.21.15.64:0/2668076998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401020f0 msgr2=0x7ffb40102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 --2- 172.21.15.64:0/2668076998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401020f0 0x7ffb40102570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009990 tx=0x7ffb34009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 -- 172.21.15.64:0/2668076998 shutdown_connections 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 --2- 172.21.15.64:0/2668076998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb40102ab0 0x7ffb401093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 --2- 172.21.15.64:0/2668076998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb40100f20 0x7ffb40101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 --2- 172.21.15.64:0/2668076998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401020f0 0x7ffb40102570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.472+0000 7ffb48211700 1 -- 172.21.15.64:0/2668076998 >> 172.21.15.64:0/2668076998 conn(0x7ffb400fc4e0 msgr2=0x7ffb400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:28.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 -- 172.21.15.64:0/2668076998 shutdown_connections 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 -- 172.21.15.64:0/2668076998 wait complete. 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 Processor -- start 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 -- start start 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb40100f20 0x7ffb40196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb401020f0 0x7ffb401973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 0x7ffb4019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4019e9e0 con 0x7ffb40102ab0 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4019eb20 con 0x7ffb401020f0 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb48211700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4019ecb0 con 0x7ffb40100f20 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb467ae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 0x7ffb4019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb467ae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 0x7ffb4019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39030/0 (socket says 172.21.15.64:39030) 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.473+0000 7ffb467ae700 1 -- 172.21.15.64:0/1898627851 learned_addr learned my addr 172.21.15.64:0/1898627851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb457ac700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb401020f0 0x7ffb401973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb45fad700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb40100f20 0x7ffb40196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb467ae700 1 -- 172.21.15.64:0/1898627851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb40100f20 msgr2=0x7ffb40196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb467ae700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb40100f20 0x7ffb40196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb467ae700 1 -- 172.21.15.64:0/1898627851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb401020f0 msgr2=0x7ffb401973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb467ae700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb401020f0 0x7ffb401973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb467ae700 1 -- 172.21.15.64:0/1898627851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340095f0 con 0x7ffb40102ab0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb467ae700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 0x7ffb4019e410 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffb38007ee0 tx=0x7ffb3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800dd20 con 0x7ffb40102ab0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb3800f040 con 0x7ffb40102ab0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4019ef90 con 0x7ffb40102ab0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380048a0 con 0x7ffb40102ab0 2024-05-08T21:54:28.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.474+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4019f410 con 0x7ffb40102ab0 2024-05-08T21:54:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.475+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40108410 con 0x7ffb40102ab0 2024-05-08T21:54:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.476+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ffb38003a00 con 0x7ffb40102ab0 2024-05-08T21:54:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.477+0000 7ffb32ffd700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffb2c0a1230 0x7ffb2c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.477+0000 7ffb45fad700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffb2c0a1230 0x7ffb2c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.477+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffb380cca90 con 0x7ffb40102ab0 2024-05-08T21:54:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.477+0000 7ffb45fad700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffb2c0a1230 0x7ffb2c0a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb40100c40 tx=0x7ffb3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:28.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.480+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffb38096070 con 0x7ffb40102ab0 2024-05-08T21:54:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.688+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb4010c790 con 0x7ffb40102ab0 2024-05-08T21:54:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.688+0000 7ffb32ffd700 1 -- 172.21.15.64:0/1898627851 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb38023020 con 0x7ffb40102ab0 2024-05-08T21:54:28.689 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:54:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.690+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffb2c0a1230 msgr2=0x7ffb2c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffb2c0a1230 0x7ffb2c0a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb40100c40 tx=0x7ffb3c00b410 comp rx=0 tx=0).stop 2024-05-08T21:54:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 msgr2=0x7ffb4019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 0x7ffb4019e410 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffb38007ee0 tx=0x7ffb3800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 shutdown_connections 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ffb2c0a1230 0x7ffb2c0a36f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb40102ab0 0x7ffb4019e410 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb40100f20 0x7ffb40196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 --2- 172.21.15.64:0/1898627851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb401020f0 0x7ffb401973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 >> 172.21.15.64:0/1898627851 conn(0x7ffb400fc4e0 msgr2=0x7ffb40105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 shutdown_connections 2024-05-08T21:54:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.691+0000 7ffb48211700 1 -- 172.21.15.64:0/1898627851 wait complete. 2024-05-08T21:54:28.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:54:28.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 -- 172.21.15.64:0/2700416485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200ff870 msgr2=0x7fc6200ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 --2- 172.21.15.64:0/2700416485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200ff870 0x7fc6200ffc90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc610009990 tx=0x7fc610009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 -- 172.21.15.64:0/2700416485 shutdown_connections 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 --2- 172.21.15.64:0/2700416485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc620100b90 0x7fc620101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 --2- 172.21.15.64:0/2700416485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200ff870 0x7fc6200ffc90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 --2- 172.21.15.64:0/2700416485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6201001d0 0x7fc620100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 -- 172.21.15.64:0/2700416485 >> 172.21.15.64:0/2700416485 conn(0x7fc6200fb3f0 msgr2=0x7fc6200fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 -- 172.21.15.64:0/2700416485 shutdown_connections 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.833+0000 7fc62511c700 1 -- 172.21.15.64:0/2700416485 wait complete. 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.834+0000 7fc62511c700 1 Processor -- start 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.834+0000 7fc62511c700 1 -- start start 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.834+0000 7fc62511c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6200ff870 0x7fc6201990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.834+0000 7fc62511c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6201001d0 0x7fc6201995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.834+0000 7fc62511c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 0x7fc6201a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.834+0000 7fc62511c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a0be0 con 0x7fc620100b90 2024-05-08T21:54:28.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc62511c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a0d20 con 0x7fc6201001d0 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc62511c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a0eb0 con 0x7fc6200ff870 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 0x7fc6201a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6200ff870 0x7fc6201990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 0x7fc6201a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39042/0 (socket says 172.21.15.64:39042) 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 -- 172.21.15.64:0/3501965155 learned_addr learned my addr 172.21.15.64:0/3501965155 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61e59c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6201001d0 0x7fc6201995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 -- 172.21.15.64:0/3501965155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6200ff870 msgr2=0x7fc6201990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6200ff870 0x7fc6201990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 -- 172.21.15.64:0/3501965155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6201001d0 msgr2=0x7fc6201995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6201001d0 0x7fc6201995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 -- 172.21.15.64:0/3501965155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6100095f0 con 0x7fc620100b90 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc61f59e700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 0x7fc6201a0610 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc618007d70 tx=0x7fc61800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:28.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800ad40 con 0x7fc620100b90 2024-05-08T21:54:28.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6201a1190 con 0x7fc620100b90 2024-05-08T21:54:28.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6180128e0 con 0x7fc620100b90 2024-05-08T21:54:28.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180116b0 con 0x7fc620100b90 2024-05-08T21:54:28.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.835+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6201a16a0 con 0x7fc620100b90 2024-05-08T21:54:28.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.836+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62010a880 con 0x7fc620100b90 2024-05-08T21:54:28.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.838+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc618012420 con 0x7fc620100b90 2024-05-08T21:54:28.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.838+0000 7fc607fff700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc6080a11b0 0x7fc6080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:28.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.838+0000 7fc61ed9d700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc6080a11b0 0x7fc6080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:28.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.838+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc6180ccb70 con 0x7fc620100b90 2024-05-08T21:54:28.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.839+0000 7fc61ed9d700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc6080a11b0 0x7fc6080a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b5c0 tx=0x7fc610003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:28.843+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6180962c0 con 0x7fc620100b90 2024-05-08T21:54:28.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1721894956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1898627851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1721894956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1898627851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.050+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc62010ead0 con 0x7fc620100b90 2024-05-08T21:54:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.051+0000 7fc607fff700 1 -- 172.21.15.64:0/3501965155 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6180998e0 con 0x7fc620100b90 2024-05-08T21:54:29.051 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-08T21:54:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc6080a11b0 msgr2=0x7fc6080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc6080a11b0 0x7fc6080a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b5c0 tx=0x7fc610003790 comp rx=0 tx=0).stop 2024-05-08T21:54:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 msgr2=0x7fc6201a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 0x7fc6201a0610 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc618007d70 tx=0x7fc61800a340 comp rx=0 tx=0).stop 2024-05-08T21:54:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 shutdown_connections 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc6080a11b0 0x7fc6080a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc620100b90 0x7fc6201a0610 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6200ff870 0x7fc6201990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 --2- 172.21.15.64:0/3501965155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6201001d0 0x7fc6201995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 >> 172.21.15.64:0/3501965155 conn(0x7fc6200fb3f0 msgr2=0x7fc6200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 shutdown_connections 2024-05-08T21:54:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.053+0000 7fc62511c700 1 -- 172.21.15.64:0/3501965155 wait complete. 2024-05-08T21:54:29.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:54:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1721894956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:29.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1898627851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 -- 172.21.15.64:0/4140713486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc101970 msgr2=0x7f61cc108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 --2- 172.21.15.64:0/4140713486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc101970 0x7f61cc108290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61c400b0a0 tx=0x7f61c400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 -- 172.21.15.64:0/4140713486 shutdown_connections 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 --2- 172.21.15.64:0/4140713486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0ffe60 0x7f61cc100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 --2- 172.21.15.64:0/4140713486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc101970 0x7f61cc108290 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 --2- 172.21.15.64:0/4140713486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61cc100fb0 0x7f61cc101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 -- 172.21.15.64:0/4140713486 >> 172.21.15.64:0/4140713486 conn(0x7f61cc0fb3f0 msgr2=0x7f61cc0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 -- 172.21.15.64:0/4140713486 shutdown_connections 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.197+0000 7f61d24a8700 1 -- 172.21.15.64:0/4140713486 wait complete. 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 Processor -- start 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 -- start start 2024-05-08T21:54:29.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0ffe60 0x7f61cc1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 0x7f61cc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61cc101970 0x7f61cc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc19eb90 con 0x7f61cc100fb0 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc19ecd0 con 0x7f61cc0ffe60 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61d24a8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc19ee60 con 0x7f61cc101970 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 0x7f61cc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 0x7f61cc1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39058/0 (socket says 172.21.15.64:39058) 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 -- 172.21.15.64:0/943890447 learned_addr learned my addr 172.21.15.64:0/943890447 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cbfff700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0ffe60 0x7f61cc1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 -- 172.21.15.64:0/943890447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61cc101970 msgr2=0x7f61cc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61cc101970 0x7f61cc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 -- 172.21.15.64:0/943890447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0ffe60 msgr2=0x7f61cc1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0ffe60 0x7f61cc1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.198+0000 7f61cb7fe700 1 -- 172.21.15.64:0/943890447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4009d00 con 0x7f61cc100fb0 2024-05-08T21:54:29.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.199+0000 7f61cb7fe700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 0x7f61cc1975f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f61c000d950 tx=0x7f61c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:29.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.199+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0009940 con 0x7f61cc100fb0 2024-05-08T21:54:29.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.199+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61c0010460 con 0x7f61cc100fb0 2024-05-08T21:54:29.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.199+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc19f140 con 0x7f61cc100fb0 2024-05-08T21:54:29.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.199+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000f620 con 0x7f61cc100fb0 2024-05-08T21:54:29.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.199+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc19f650 con 0x7f61cc100fb0 2024-05-08T21:54:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.200+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc067cf0 con 0x7f61cc100fb0 2024-05-08T21:54:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.201+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f61c00108d0 con 0x7f61cc100fb0 2024-05-08T21:54:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.202+0000 7f61c97fa700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f61b40a11b0 0x7f61b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.202+0000 7f61cbfff700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f61b40a11b0 0x7f61b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.202+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f61c00ccf50 con 0x7f61cc100fb0 2024-05-08T21:54:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.202+0000 7f61cbfff700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f61b40a11b0 0x7f61b40a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009c60 tx=0x7f61bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.205+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f61c0096590 con 0x7f61cc100fb0 2024-05-08T21:54:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.416+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61cc19f930 con 0x7f61cc100fb0 2024-05-08T21:54:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.417+0000 7f61c97fa700 1 -- 172.21.15.64:0/943890447 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61c0094e60 con 0x7f61cc100fb0 2024-05-08T21:54:29.417 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-08T21:54:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f61b40a11b0 msgr2=0x7f61b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f61b40a11b0 0x7f61b40a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009c60 tx=0x7f61bc009260 comp rx=0 tx=0).stop 2024-05-08T21:54:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 msgr2=0x7f61cc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 0x7f61cc1975f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f61c000d950 tx=0x7f61c000dc60 comp rx=0 tx=0).stop 2024-05-08T21:54:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 shutdown_connections 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f61b40a11b0 0x7f61b40a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0ffe60 0x7f61cc1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61cc101970 0x7f61cc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 --2- 172.21.15.64:0/943890447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61cc100fb0 0x7f61cc1975f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 >> 172.21.15.64:0/943890447 conn(0x7f61cc0fb3f0 msgr2=0x7f61cc0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 shutdown_connections 2024-05-08T21:54:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.419+0000 7f61d24a8700 1 -- 172.21.15.64:0/943890447 wait complete. 2024-05-08T21:54:29.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 -- 172.21.15.64:0/1119086453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0106b70 msgr2=0x7f84b0108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 --2- 172.21.15.64:0/1119086453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0106b70 0x7f84b0108fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b0a0 tx=0x7f84a800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 -- 172.21.15.64:0/1119086453 shutdown_connections 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 --2- 172.21.15.64:0/1119086453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0101910 0x7f84b0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 --2- 172.21.15.64:0/1119086453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0106b70 0x7f84b0108fe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 --2- 172.21.15.64:0/1119086453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b0104240 0x7f84b0106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 -- 172.21.15.64:0/1119086453 >> 172.21.15.64:0/1119086453 conn(0x7f84b00fb320 msgr2=0x7f84b00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 -- 172.21.15.64:0/1119086453 shutdown_connections 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 -- 172.21.15.64:0/1119086453 wait complete. 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.558+0000 7f84b754b700 1 Processor -- start 2024-05-08T21:54:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 -- start start 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 0x7f84b0196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b0104240 0x7f84b01973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0106b70 0x7f84b019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b019e9e0 con 0x7f84b0101910 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b019eb20 con 0x7f84b0106b70 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b754b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b019ecb0 con 0x7f84b0104240 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 0x7f84b0196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 0x7f84b0196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39076/0 (socket says 172.21.15.64:39076) 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 -- 172.21.15.64:0/2946242920 learned_addr learned my addr 172.21.15.64:0/2946242920 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b4ae6700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b0104240 0x7f84b01973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b5ae8700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0106b70 0x7f84b019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 -- 172.21.15.64:0/2946242920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b0104240 msgr2=0x7f84b01973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b0104240 0x7f84b01973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 -- 172.21.15.64:0/2946242920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0106b70 msgr2=0x7f84b019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0106b70 0x7f84b019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 -- 172.21.15.64:0/2946242920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8009d00 con 0x7f84b0101910 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b5ae8700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0106b70 0x7f84b019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84b52e7700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 0x7f84b0196eb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00d950 tx=0x7f84ac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.559+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac009b40 con 0x7f84b0101910 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.560+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac009ca0 con 0x7f84b0101910 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.560+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b019ef90 con 0x7f84b0101910 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.560+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00f620 con 0x7f84b0101910 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.560+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b019f3b0 con 0x7f84b0101910 2024-05-08T21:54:29.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.560+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b00fcea0 con 0x7f84b0101910 2024-05-08T21:54:29.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.562+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f84ac010460 con 0x7f84b0101910 2024-05-08T21:54:29.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.563+0000 7f84a67fc700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f849c0a11b0 0x7f849c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.563+0000 7f84b4ae6700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f849c0a11b0 0x7f849c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.563+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84ac0ccde0 con 0x7f84b0101910 2024-05-08T21:54:29.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.563+0000 7f84b4ae6700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f849c0a11b0 0x7f849c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84a0005950 tx=0x7f84a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:29.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.566+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84ac096420 con 0x7f84b0101910 2024-05-08T21:54:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.824+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84b010c6c0 con 0x7f84b0101910 2024-05-08T21:54:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.824+0000 7f84a67fc700 1 -- 172.21.15.64:0/2946242920 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10921 (secure 0 0 0) 0x7f84ac020090 con 0x7f84b0101910 2024-05-08T21:54:29.825 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:29.825 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T21:53:29.959964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.826+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f849c0a11b0 msgr2=0x7f849c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.826+0000 7f84b754b700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f849c0a11b0 0x7f849c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84a0005950 tx=0x7f84a00058e0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.826+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 msgr2=0x7f84b0196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.826+0000 7f84b754b700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 0x7f84b0196eb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00d950 tx=0x7f84ac00dd10 comp rx=0 tx=0).stop 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 shutdown_connections 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f849c0a11b0 0x7f849c0a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84b0106b70 0x7f84b019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0101910 0x7f84b0196eb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 --2- 172.21.15.64:0/2946242920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b0104240 0x7f84b01973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 >> 172.21.15.64:0/2946242920 conn(0x7f84b00fb320 msgr2=0x7f84b0102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 shutdown_connections 2024-05-08T21:54:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.827+0000 7f84b754b700 1 -- 172.21.15.64:0/2946242920 wait complete. 2024-05-08T21:54:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:54:29.841 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:54:29.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:29 smithi064 ceph-mon[42500]: pgmap v255: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:29.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:29 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3501965155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:29.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:29 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/943890447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/122201682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f94101910 msgr2=0x7f0f941081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/122201682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f94101910 0x7f0f941081c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00b0d0 tx=0x7f0f8c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/122201682 shutdown_connections 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/122201682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f940ffd80 0x7f0f941001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/122201682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f94101910 0x7f0f941081c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/122201682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94100f70 0x7f0f941013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/122201682 >> 172.21.15.64:0/122201682 conn(0x7f0f940fb2f0 msgr2=0x7f0f940fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/122201682 shutdown_connections 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/122201682 wait complete. 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 Processor -- start 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.969+0000 7f0f9a7d9700 1 -- start start 2024-05-08T21:54:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 0x7f0f94196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f94100f70 0x7f0f941974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94101910 0x7f0f9419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9419ea50 con 0x7f0f940ffd80 2024-05-08T21:54:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9419eb90 con 0x7f0f94100f70 2024-05-08T21:54:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9419ed20 con 0x7f0f94101910 2024-05-08T21:54:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 0x7f0f94196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 0x7f0f94196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39088/0 (socket says 172.21.15.64:39088) 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 -- 172.21.15.64:0/3221090382 learned_addr learned my addr 172.21.15.64:0/3221090382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f98d76700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94101910 0x7f0f9419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 -- 172.21.15.64:0/3221090382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94101910 msgr2=0x7f0f9419e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f8bfff700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f94100f70 0x7f0f941974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94101910 0x7f0f9419e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 -- 172.21.15.64:0/3221090382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f94100f70 msgr2=0x7f0f941974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f94100f70 0x7f0f941974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 -- 172.21.15.64:0/3221090382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c009d00 con 0x7f0f940ffd80 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f98d76700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94101910 0x7f0f9419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f93fff700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 0x7f0f94196f70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400d980 tx=0x7f0f8400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400de60 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f840044e0 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9419efa0 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84003be0 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.970+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9419f470 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.971+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f94067380 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.973+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0f84009bb0 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.973+0000 7f0f91ffb700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0f800a1230 0x7f0f800a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.973+0000 7f0f8bfff700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0f800a1230 0x7f0f800a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.973+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f840ccac0 con 0x7f0f940ffd80 2024-05-08T21:54:29.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.974+0000 7f0f8bfff700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0f800a1230 0x7f0f800a36f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00ba60 tx=0x7f0f7c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:29.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:29.977+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f84096180 con 0x7f0f940ffd80 2024-05-08T21:54:30.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:29 smithi160 ceph-mon[46962]: pgmap v255: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:30.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:29 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3501965155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:30.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:29 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/943890447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:29 smithi186 ceph-mon[46619]: pgmap v255: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:29 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3501965155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:29 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/943890447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:30.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.187+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f9410b5f0 con 0x7f0f940ffd80 2024-05-08T21:54:30.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.189+0000 7f0f91ffb700 1 -- 172.21.15.64:0/3221090382 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22495 (secure 0 0 0) 0x7f0f84017090 con 0x7f0f940ffd80 2024-05-08T21:54:30.189 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:30.190 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":81,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:54:21.242448+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0f800a1230 msgr2=0x7f0f800a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0f800a1230 0x7f0f800a36f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00ba60 tx=0x7f0f7c005d50 comp rx=0 tx=0).stop 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 msgr2=0x7f0f94196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 0x7f0f94196f70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400d980 tx=0x7f0f8400dd40 comp rx=0 tx=0).stop 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 shutdown_connections 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0f800a1230 0x7f0f800a36f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94101910 0x7f0f9419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f940ffd80 0x7f0f94196f70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 --2- 172.21.15.64:0/3221090382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f94100f70 0x7f0f941974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 >> 172.21.15.64:0/3221090382 conn(0x7f0f940fb2f0 msgr2=0x7f0f940fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 shutdown_connections 2024-05-08T21:54:30.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.191+0000 7f0f9a7d9700 1 -- 172.21.15.64:0/3221090382 wait complete. 2024-05-08T21:54:30.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:53:21.578075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:53:22.933075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:54:30.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:54:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1288735240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8106c40 msgr2=0x7fa6f81090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1288735240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8106c40 0x7fa6f81090b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000b0a0 tx=0x7fa6f000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1288735240 shutdown_connections 2024-05-08T21:54:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1288735240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8106c40 0x7fa6f81090b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1288735240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1288735240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6f8104310 0x7fa6f8106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1288735240 >> 172.21.15.64:0/1288735240 conn(0x7fa6f80fb3f0 msgr2=0x7fa6f80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1288735240 shutdown_connections 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.335+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1288735240 wait complete. 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 Processor -- start 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 -- start start 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8104310 0x7fa6f81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6f8106c40 0x7fa6f81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a0cd0 con 0x7fa6f8104310 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a0e10 con 0x7fa6f8106c40 2024-05-08T21:54:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6ffd8a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a0fa0 con 0x7fa6f81019e0 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fdb26700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fdb26700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48248/0 (socket says 172.21.15.64:48248) 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fdb26700 1 -- 172.21.15.64:0/1049449825 learned_addr learned my addr 172.21.15.64:0/1049449825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fd325700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8104310 0x7fa6f81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fe327700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6f8106c40 0x7fa6f81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fd325700 1 -- 172.21.15.64:0/1049449825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 msgr2=0x7fa6f81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fd325700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fd325700 1 -- 172.21.15.64:0/1049449825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6f8106c40 msgr2=0x7fa6f81a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fd325700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6f8106c40 0x7fa6f81a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fd325700 1 -- 172.21.15.64:0/1049449825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0009d00 con 0x7fa6f8104310 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.336+0000 7fa6fdb26700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.337+0000 7fa6fd325700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8104310 0x7fa6f81996e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400d950 tx=0x7fa6f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.337+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400f890 con 0x7fa6f8104310 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.337+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6f4010460 con 0x7fa6f8104310 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.337+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f81a1280 con 0x7fa6f8104310 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.337+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400e610 con 0x7fa6f8104310 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.337+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81a1790 con 0x7fa6f8104310 2024-05-08T21:54:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.338+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f81087c0 con 0x7fa6f8104310 2024-05-08T21:54:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.339+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa6f40108d0 con 0x7fa6f8104310 2024-05-08T21:54:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.339+0000 7fa6eaffd700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6e40a11b0 0x7fa6e40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.340+0000 7fa6fdb26700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6e40a11b0 0x7fa6e40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.340+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa6f40ccb70 con 0x7fa6f8104310 2024-05-08T21:54:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.340+0000 7fa6fdb26700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6e40a11b0 0x7fa6e40a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0104e0 tx=0x7fa6ec007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:30.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.343+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa6f40961b0 con 0x7fa6f8104310 2024-05-08T21:54:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.548+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6f810c850 con 0x7fa6f8104310 2024-05-08T21:54:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.549+0000 7fa6eaffd700 1 -- 172.21.15.64:0/1049449825 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6f40160a0 con 0x7fa6f8104310 2024-05-08T21:54:30.549 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T21:54:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6e40a11b0 msgr2=0x7fa6e40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6e40a11b0 0x7fa6e40a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0104e0 tx=0x7fa6ec007480 comp rx=0 tx=0).stop 2024-05-08T21:54:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8104310 msgr2=0x7fa6f81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8104310 0x7fa6f81996e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400d950 tx=0x7fa6f400dc60 comp rx=0 tx=0).stop 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 shutdown_connections 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6e40a11b0 0x7fa6e40a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6f8106c40 0x7fa6f81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6f81019e0 0x7fa6f81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 --2- 172.21.15.64:0/1049449825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6f8104310 0x7fa6f81996e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 >> 172.21.15.64:0/1049449825 conn(0x7fa6f80fb3f0 msgr2=0x7fa6f8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 shutdown_connections 2024-05-08T21:54:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.551+0000 7fa6ffd8a700 1 -- 172.21.15.64:0/1049449825 wait complete. 2024-05-08T21:54:30.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 -- 172.21.15.64:0/977480176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd828102ab0 msgr2=0x7fd8281093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 --2- 172.21.15.64:0/977480176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd828102ab0 0x7fd8281093d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd82000b0a0 tx=0x7fd82000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 -- 172.21.15.64:0/977480176 shutdown_connections 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 --2- 172.21.15.64:0/977480176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd828102ab0 0x7fd8281093d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 --2- 172.21.15.64:0/977480176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd828100f20 0x7fd828101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 --2- 172.21.15.64:0/977480176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8281020f0 0x7fd828102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 -- 172.21.15.64:0/977480176 >> 172.21.15.64:0/977480176 conn(0x7fd8280fc4e0 msgr2=0x7fd8280fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:30.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 -- 172.21.15.64:0/977480176 shutdown_connections 2024-05-08T21:54:30.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.693+0000 7fd82db6e700 1 -- 172.21.15.64:0/977480176 wait complete. 2024-05-08T21:54:30.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 Processor -- start 2024-05-08T21:54:30.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 -- start start 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd828100f20 0x7fd828074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 0x7fd828073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd828102ab0 0x7fd8280738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828074070 con 0x7fd8281020f0 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280741e0 con 0x7fd828100f20 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd82db6e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281a0ff0 con 0x7fd828102ab0 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 0x7fd828073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 0x7fd828073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39112/0 (socket says 172.21.15.64:39112) 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 -- 172.21.15.64:0/1604735677 learned_addr learned my addr 172.21.15.64:0/1604735677 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd8277fe700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd828100f20 0x7fd828074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd827fff700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd828102ab0 0x7fd8280738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 -- 172.21.15.64:0/1604735677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd828102ab0 msgr2=0x7fd8280738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd828102ab0 0x7fd8280738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 -- 172.21.15.64:0/1604735677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd828100f20 msgr2=0x7fd828074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.694+0000 7fd826ffd700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd828100f20 0x7fd828074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd826ffd700 1 -- 172.21.15.64:0/1604735677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd820009d00 con 0x7fd8281020f0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd826ffd700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 0x7fd828073390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d950 tx=0x7fd81c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00f890 con 0x7fd8281020f0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c010460 con 0x7fd8281020f0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8281a1280 con 0x7fd8281020f0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0195c0 con 0x7fd8281020f0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.695+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8281a17e0 con 0x7fd8281020f0 2024-05-08T21:54:30.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.696+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd828108340 con 0x7fd8281020f0 2024-05-08T21:54:30.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.698+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd81c0108d0 con 0x7fd8281020f0 2024-05-08T21:54:30.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.698+0000 7fd824ff9700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8100a1230 0x7fd8100a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:30.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.699+0000 7fd8277fe700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8100a1230 0x7fd8100a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:30.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.699+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd81c0cc800 con 0x7fd8281020f0 2024-05-08T21:54:30.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.699+0000 7fd8277fe700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8100a1230 0x7fd8100a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd828100c40 tx=0x7fd81800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:30.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.702+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd81c095de0 con 0x7fd8281020f0 2024-05-08T21:54:30.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.909+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd82810c790 con 0x7fd8281020f0 2024-05-08T21:54:30.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.910+0000 7fd824ff9700 1 -- 172.21.15.64:0/1604735677 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd81c020020 con 0x7fd8281020f0 2024-05-08T21:54:30.911 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8100a1230 msgr2=0x7fd8100a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8100a1230 0x7fd8100a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd828100c40 tx=0x7fd81800a3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 msgr2=0x7fd828073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 0x7fd828073390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d950 tx=0x7fd81c00dc60 comp rx=0 tx=0).stop 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 shutdown_connections 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd8100a1230 0x7fd8100a36f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd828100f20 0x7fd828074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd828102ab0 0x7fd8280738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 --2- 172.21.15.64:0/1604735677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8281020f0 0x7fd828073390 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 >> 172.21.15.64:0/1604735677 conn(0x7fd8280fc4e0 msgr2=0x7fd828105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:30.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 shutdown_connections 2024-05-08T21:54:30.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:30.912+0000 7fd82db6e700 1 -- 172.21.15.64:0/1604735677 wait complete. 2024-05-08T21:54:30.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:30 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2946242920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:30.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:30 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3221090382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:30.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:30 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1049449825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:30.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:54:31.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:30 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2946242920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:31.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:30 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3221090382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:31.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:30 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1049449825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:31.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 -- 172.21.15.64:0/1854542955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc1007d0 msgr2=0x7f84dc100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 --2- 172.21.15.64:0/1854542955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc1007d0 0x7f84dc100c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f84c80099c0 tx=0x7f84c8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 -- 172.21.15.64:0/1854542955 shutdown_connections 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 --2- 172.21.15.64:0/1854542955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84dc101170 0x7f84dc107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 --2- 172.21.15.64:0/1854542955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84dc0ff5e0 0x7f84dc0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 --2- 172.21.15.64:0/1854542955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc1007d0 0x7f84dc100c30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 -- 172.21.15.64:0/1854542955 >> 172.21.15.64:0/1854542955 conn(0x7f84dc0fab50 msgr2=0x7f84dc0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 -- 172.21.15.64:0/1854542955 shutdown_connections 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.054+0000 7f84e321d700 1 -- 172.21.15.64:0/1854542955 wait complete. 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 Processor -- start 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 -- start start 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84dc0ff5e0 0x7f84dc196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84dc1007d0 0x7f84dc194ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 0x7f84dc195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1957c0 con 0x7f84dc101170 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc195930 con 0x7f84dc0ff5e0 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e321d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1a0f70 con 0x7f84dc1007d0 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 0x7f84dc195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 0x7f84dc195020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39118/0 (socket says 172.21.15.64:39118) 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 -- 172.21.15.64:0/3180961854 learned_addr learned my addr 172.21.15.64:0/3180961854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e0fb9700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84dc0ff5e0 0x7f84dc196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 -- 172.21.15.64:0/3180961854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84dc1007d0 msgr2=0x7f84dc194ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84dc1007d0 0x7f84dc194ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 -- 172.21.15.64:0/3180961854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84dc0ff5e0 msgr2=0x7f84dc196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84dc0ff5e0 0x7f84dc196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.055+0000 7f84e17ba700 1 -- 172.21.15.64:0/3180961854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80095f0 con 0x7f84dc101170 2024-05-08T21:54:31.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.056+0000 7f84e17ba700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 0x7f84dc195020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009fa0 tx=0x7f84d40076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:31.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.056+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4003c20 con 0x7f84dc101170 2024-05-08T21:54:31.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.056+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84d4010040 con 0x7f84dc101170 2024-05-08T21:54:31.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.056+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc1a1200 con 0x7f84dc101170 2024-05-08T21:54:31.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.056+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4014650 con 0x7f84dc101170 2024-05-08T21:54:31.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.056+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc1a16c0 con 0x7f84dc101170 2024-05-08T21:54:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.057+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc106a10 con 0x7f84dc101170 2024-05-08T21:54:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.058+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f84d4004030 con 0x7f84dc101170 2024-05-08T21:54:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.059+0000 7f84d9ffb700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f84cc0a11b0 0x7f84cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.059+0000 7f84e0fb9700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f84cc0a11b0 0x7f84cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.059+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84d40cc620 con 0x7f84dc101170 2024-05-08T21:54:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.059+0000 7f84e0fb9700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f84cc0a11b0 0x7f84cc0a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84dc100630 tx=0x7f84d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:31.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.063+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84d4095c60 con 0x7f84dc101170 2024-05-08T21:54:31.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:30 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2946242920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:31.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:30 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3221090382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:31.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:30 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1049449825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:54:31.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.273+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84dc10a660 con 0x7f84dc101170 2024-05-08T21:54:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.273+0000 7f84d9ffb700 1 -- 172.21.15.64:0/3180961854 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84d40840a0 con 0x7f84dc101170 2024-05-08T21:54:31.274 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f84cc0a11b0 msgr2=0x7f84cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f84cc0a11b0 0x7f84cc0a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84dc100630 tx=0x7f84d000b410 comp rx=0 tx=0).stop 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 msgr2=0x7f84dc195020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 0x7f84dc195020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009fa0 tx=0x7f84d40076c0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 shutdown_connections 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f84cc0a11b0 0x7f84cc0a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84dc0ff5e0 0x7f84dc196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84dc101170 0x7f84dc195020 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 --2- 172.21.15.64:0/3180961854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84dc1007d0 0x7f84dc194ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.275+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 >> 172.21.15.64:0/3180961854 conn(0x7f84dc0fab50 msgr2=0x7f84dc103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.276+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 shutdown_connections 2024-05-08T21:54:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.276+0000 7f84e321d700 1 -- 172.21.15.64:0/3180961854 wait complete. 2024-05-08T21:54:31.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 -- 172.21.15.64:0/4202748451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc100f50 msgr2=0x7efffc1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 --2- 172.21.15.64:0/4202748451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc100f50 0x7efffc1013b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efff00099c0 tx=0x7efff0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 -- 172.21.15.64:0/4202748451 shutdown_connections 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 --2- 172.21.15.64:0/4202748451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efffc1018f0 0x7efffc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 --2- 172.21.15.64:0/4202748451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efffc0ffd60 0x7efffc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 --2- 172.21.15.64:0/4202748451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc100f50 0x7efffc1013b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 -- 172.21.15.64:0/4202748451 >> 172.21.15.64:0/4202748451 conn(0x7efffc0fb2f0 msgr2=0x7efffc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 -- 172.21.15.64:0/4202748451 shutdown_connections 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.415+0000 7f0001a19700 1 -- 172.21.15.64:0/4202748451 wait complete. 2024-05-08T21:54:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 Processor -- start 2024-05-08T21:54:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 -- start start 2024-05-08T21:54:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 0x7efffc196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efffc100f50 0x7efffc197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efffc1018f0 0x7efffc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc19e930 con 0x7efffc0ffd60 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc19ea70 con 0x7efffc100f50 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7f0001a19700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc19ec00 con 0x7efffc1018f0 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 0x7efffc196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 0x7efffc196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39122/0 (socket says 172.21.15.64:39122) 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 -- 172.21.15.64:0/2958565119 learned_addr learned my addr 172.21.15.64:0/2958565119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffa7fc700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efffc100f50 0x7efffc197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 -- 172.21.15.64:0/2958565119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efffc1018f0 msgr2=0x7efffc19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efffc1018f0 0x7efffc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 -- 172.21.15.64:0/2958565119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efffc100f50 msgr2=0x7efffc197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.416+0000 7efffaffd700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efffc100f50 0x7efffc197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7efffaffd700 1 -- 172.21.15.64:0/2958565119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff00095f0 con 0x7efffc0ffd60 2024-05-08T21:54:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7efffaffd700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 0x7efffc196e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7effec00d980 tx=0x7effec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec004020 con 0x7efffc0ffd60 2024-05-08T21:54:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effec009730 con 0x7efffc0ffd60 2024-05-08T21:54:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc19eee0 con 0x7efffc0ffd60 2024-05-08T21:54:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec003a80 con 0x7efffc0ffd60 2024-05-08T21:54:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.417+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc19f300 con 0x7efffc0ffd60 2024-05-08T21:54:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.418+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc107130 con 0x7efffc0ffd60 2024-05-08T21:54:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.419+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7effec0098a0 con 0x7efffc0ffd60 2024-05-08T21:54:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.419+0000 7f0000a17700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7effe40a11b0 0x7effe40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.420+0000 7efffa7fc700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7effe40a11b0 0x7effe40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.420+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effec0cc760 con 0x7efffc0ffd60 2024-05-08T21:54:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.420+0000 7efffa7fc700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7effe40a11b0 0x7effe40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff00037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.424+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7effec095da0 con 0x7efffc0ffd60 2024-05-08T21:54:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.678+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efffc067c10 con 0x7efffc0ffd60 2024-05-08T21:54:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.679+0000 7f0000a17700 1 -- 172.21.15.64:0/2958565119 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10921 (secure 0 0 0) 0x7effec0993c0 con 0x7efffc0ffd60 2024-05-08T21:54:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:31.680 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T21:53:29.959964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7effe40a11b0 msgr2=0x7effe40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7effe40a11b0 0x7effe40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff00037c0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 msgr2=0x7efffc196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 0x7efffc196e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7effec00d980 tx=0x7effec00dc90 comp rx=0 tx=0).stop 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 shutdown_connections 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7effe40a11b0 0x7effe40a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efffc1018f0 0x7efffc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efffc0ffd60 0x7efffc196e00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 --2- 172.21.15.64:0/2958565119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efffc100f50 0x7efffc197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 >> 172.21.15.64:0/2958565119 conn(0x7efffc0fb2f0 msgr2=0x7efffc104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 shutdown_connections 2024-05-08T21:54:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.681+0000 7f0001a19700 1 -- 172.21.15.64:0/2958565119 wait complete. 2024-05-08T21:54:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-08T21:54:31.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 -- 172.21.15.64:0/2272152773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2301018f0 msgr2=0x7ff2301081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 --2- 172.21.15.64:0/2272152773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2301018f0 0x7ff2301081a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff22800b0d0 tx=0x7ff22800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 -- 172.21.15.64:0/2272152773 shutdown_connections 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 --2- 172.21.15.64:0/2272152773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2300ffd60 0x7ff230100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 --2- 172.21.15.64:0/2272152773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2301018f0 0x7ff2301081a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 --2- 172.21.15.64:0/2272152773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff230100f50 0x7ff2301013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 -- 172.21.15.64:0/2272152773 >> 172.21.15.64:0/2272152773 conn(0x7ff2300fb2f0 msgr2=0x7ff2300fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 -- 172.21.15.64:0/2272152773 shutdown_connections 2024-05-08T21:54:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.830+0000 7ff237daa700 1 -- 172.21.15.64:0/2272152773 wait complete. 2024-05-08T21:54:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 Processor -- start 2024-05-08T21:54:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 -- start start 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2300ffd60 0x7ff230195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 0x7ff2301962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2301018f0 0x7ff23019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23019d8e0 con 0x7ff230100f50 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23019da20 con 0x7ff2301018f0 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff237daa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23019dbb0 con 0x7ff2300ffd60 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 0x7ff2301962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 0x7ff2301962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39140/0 (socket says 172.21.15.64:39140) 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 -- 172.21.15.64:0/1840722267 learned_addr learned my addr 172.21.15.64:0/1840722267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff236347700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2301018f0 0x7ff23019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235b46700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2300ffd60 0x7ff230195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 -- 172.21.15.64:0/1840722267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2300ffd60 msgr2=0x7ff230195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2300ffd60 0x7ff230195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 -- 172.21.15.64:0/1840722267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2301018f0 msgr2=0x7ff23019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2301018f0 0x7ff23019d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff235345700 1 -- 172.21.15.64:0/1840722267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228009d00 con 0x7ff230100f50 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.831+0000 7ff236347700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2301018f0 0x7ff23019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.832+0000 7ff235345700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 0x7ff2301962f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e9a0 tx=0x7ff22000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.832+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220009790 con 0x7ff230100f50 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.832+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff230068050 con 0x7ff230100f50 2024-05-08T21:54:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.832+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff22000cbd0 con 0x7ff230100f50 2024-05-08T21:54:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.832+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220019840 con 0x7ff230100f50 2024-05-08T21:54:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.832+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2300685b0 con 0x7ff230100f50 2024-05-08T21:54:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.833+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff230067c10 con 0x7ff230100f50 2024-05-08T21:54:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.834+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff22000cd40 con 0x7ff230100f50 2024-05-08T21:54:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.834+0000 7ff226ffd700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff21c0a11b0 0x7ff21c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.835+0000 7ff235b46700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff21c0a11b0 0x7ff21c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.835+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2200ceb60 con 0x7ff230100f50 2024-05-08T21:54:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.835+0000 7ff235b46700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff21c0a11b0 0x7ff21c0a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff230100db0 tx=0x7ff22c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:31.838+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff220098250 con 0x7ff230100f50 2024-05-08T21:54:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:31 smithi064 ceph-mon[42500]: pgmap v256: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:31 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1604735677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:31 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3180961854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:31 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2958565119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:31 smithi160 ceph-mon[46962]: pgmap v256: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:31 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1604735677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:31 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3180961854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:31 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2958565119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:32.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.045+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff23010b720 con 0x7ff230100f50 2024-05-08T21:54:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.047+0000 7ff226ffd700 1 -- 172.21.15.64:0/1840722267 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22495 (secure 0 0 0) 0x7ff22009b870 con 0x7ff230100f50 2024-05-08T21:54:32.047 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:32.048 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":81,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:54:21.242448+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff21c0a11b0 msgr2=0x7ff21c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff21c0a11b0 0x7ff21c0a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff230100db0 tx=0x7ff22c009260 comp rx=0 tx=0).stop 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 msgr2=0x7ff2301962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 0x7ff2301962f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e9a0 tx=0x7ff22000ed60 comp rx=0 tx=0).stop 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 shutdown_connections 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff21c0a11b0 0x7ff21c0a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2301018f0 0x7ff23019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2300ffd60 0x7ff230195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 --2- 172.21.15.64:0/1840722267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff230100f50 0x7ff2301962f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 >> 172.21.15.64:0/1840722267 conn(0x7ff2300fb2f0 msgr2=0x7ff230104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 shutdown_connections 2024-05-08T21:54:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.049+0000 7ff237daa700 1 -- 172.21.15.64:0/1840722267 wait complete. 2024-05-08T21:54:32.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T21:53:23.555551+0000', 'modified': '2024-05-08T21:53:29.959964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24506, 'mds_1': 24484, 'mds_2': 24476, 'mds_3': 24427, 'mds_4': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6839/3760541428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 3760541428}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 3760541428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6841/1669365277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6840', 'nonce': 1669365277}, {'type': 'v1', 'addr': '172.21.15.64:6841', 'nonce': 1669365277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.160:6833/3542328594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6832', 'nonce': 3542328594}, {'type': 'v1', 'addr': '172.21.15.160:6833', 'nonce': 3542328594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.186:6833/1716244539', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6832', 'nonce': 1716244539}, {'type': 'v1', 'addr': '172.21.15.186:6833', 'nonce': 1716244539}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/770617410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 770617410}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 770617410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/4069999836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 4069999836}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 4069999836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/3888486173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 3888486173}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 3888486173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.160:6837/4233185014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 4233185014}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 4233185014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.186:6839/539285459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 539285459}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 539285459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/2466892222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 2466892222}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 2466892222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T21:54:32.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T21:54:32.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-05-08T21:54:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:31 smithi186 ceph-mon[46619]: pgmap v256: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:31 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1604735677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:54:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:31 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3180961854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:54:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:31 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2958565119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 -- 172.21.15.64:0/1585890612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504100f80 msgr2=0x7f65041013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 --2- 172.21.15.64:0/1585890612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504100f80 0x7f65041013a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f64f40099c0 tx=0x7f64f4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 -- 172.21.15.64:0/1585890612 shutdown_connections 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 --2- 172.21.15.64:0/1585890612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6504102b10 0x7f65041093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 --2- 172.21.15.64:0/1585890612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504100f80 0x7f65041013a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 --2- 172.21.15.64:0/1585890612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6504102170 0x7f65041025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.190+0000 7f65097be700 1 -- 172.21.15.64:0/1585890612 >> 172.21.15.64:0/1585890612 conn(0x7f65040fc4f0 msgr2=0x7f65040fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 -- 172.21.15.64:0/1585890612 shutdown_connections 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 -- 172.21.15.64:0/1585890612 wait complete. 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 Processor -- start 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 -- start start 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6504100f80 0x7f6504196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6504102170 0x7f6504197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 0x7f650419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650419ea10 con 0x7f6504102b10 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650419eb50 con 0x7f6504100f80 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.191+0000 7f65097be700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650419ece0 con 0x7f6504102170 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 0x7f650419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 0x7f650419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39170/0 (socket says 172.21.15.64:39170) 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 -- 172.21.15.64:0/3391998658 learned_addr learned my addr 172.21.15.64:0/3391998658 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6504100f80 0x7f6504196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65027fc700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6504102170 0x7f6504197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 -- 172.21.15.64:0/3391998658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6504102170 msgr2=0x7f6504197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6504102170 0x7f6504197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 -- 172.21.15.64:0/3391998658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6504100f80 msgr2=0x7f6504196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6504100f80 0x7f6504196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 -- 172.21.15.64:0/3391998658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f40095f0 con 0x7f6504102b10 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65037fe700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 0x7f650419e440 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f64fc007ee0 tx=0x7f64fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f64fbfff700 1 -- 172.21.15.64:0/3391998658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc017070 con 0x7f6504102b10 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f64fbfff700 1 -- 172.21.15.64:0/3391998658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64fc00f040 con 0x7f6504102b10 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650419efc0 con 0x7f6504102b10 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f64fbfff700 1 -- 172.21.15.64:0/3391998658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc005580 con 0x7f6504102b10 2024-05-08T21:54:32.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.192+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650419f3f0 con 0x7f6504102b10 2024-05-08T21:54:32.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.195+0000 7f64fbfff700 1 -- 172.21.15.64:0/3391998658 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f64fc00db20 con 0x7f6504102b10 2024-05-08T21:54:32.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.195+0000 7f64fbfff700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64f00a11b0 0x7f64f00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.196+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64f00a11b0 0x7f64f00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.196+0000 7f64fbfff700 1 -- 172.21.15.64:0/3391998658 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64fc0cc1c0 con 0x7f6504102b10 2024-05-08T21:54:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.196+0000 7f6515978b80 1 Processor -- start 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.196+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64f00a11b0 0x7f64f00a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f64f4003820 tx=0x7f64f4005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.196+0000 7f6515978b80 1 -- start start 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6515978b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1df910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6515978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6515978b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6515978b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c136100 con 0x56464c1df4f0 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6515978b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c133ac0 con 0x56464c1e0810 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6515978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c12f3b0 con 0x56464c1dfe50 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1df910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39186/0 (socket says 172.21.15.64:39186) 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 -- 172.21.15.64:0/3784692454 learned_addr learned my addr 172.21.15.64:0/3784692454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f65027fc700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f65037fe700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 -- 172.21.15.64:0/3784692454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 msgr2=0x56464c1e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 -- 172.21.15.64:0/3784692454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 msgr2=0x56464c1e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 -- 172.21.15.64:0/3784692454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464bfb0360 con 0x56464c1df4f0 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f65027fc700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f65037fe700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1df910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6504101fd0 tx=0x7f64f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3525bcb2ac78de8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f64f97fa700 1 -- 172.21.15.64:0/3784692454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f40038c0 con 0x56464c1df4f0 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f64f97fa700 1 -- 172.21.15.64:0/3784692454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f401e040 con 0x56464c1df4f0 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.197+0000 7f64f97fa700 1 -- 172.21.15.64:0/3784692454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400fa30 con 0x56464c1df4f0 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 -- 172.21.15.64:0/3784692454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 msgr2=0x56464c1df910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1df910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6504101fd0 tx=0x7f64f4020000 comp rx=0 tx=0).stop 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 -- 172.21.15.64:0/3784692454 shutdown_connections 2024-05-08T21:54:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1df910 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 --2- 172.21.15.64:0/3784692454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 -- 172.21.15.64:0/3784692454 >> 172.21.15.64:0/3784692454 conn(0x56464c13d1c0 msgr2=0x56464c1d9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 -- 172.21.15.64:0/3784692454 shutdown_connections 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.198+0000 7f6515978b80 1 -- 172.21.15.64:0/3784692454 wait complete. 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 -- start start 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 Processor -- start 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1f0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c0cb510 con 0x56464c1df4f0 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c0cb1b0 con 0x56464c1e0810 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464bfb2650 con 0x56464c1dfe50 2024-05-08T21:54:32.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6515978b80 10 client.0 _ll_register_callbacks cb 0x56464c0d4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1e8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39188/0 (socket says 172.21.15.64:39188) 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 -- 172.21.15.64:0/3918177801 learned_addr learned my addr 172.21.15.64:0/3918177801 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f65037fe700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f65027fc700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 -- 172.21.15.64:0/3918177801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 msgr2=0x56464c1e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 -- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 msgr2=0x56464c1f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.199+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1f0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6502ffd700 1 -- 172.21.15.64:0/3918177801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6504109d10 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1e8cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009d10 tx=0x7f64f4004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1007ce8320ef0cf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f64ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f64ea7fc700 1 -- 172.21.15.64:0/3918177801 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4003ad0 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f64ea7fc700 1 -- 172.21.15.64:0/3918177801 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64f4003c30 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56464c088220 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f64ea7fc700 1 -- 172.21.15.64:0/3918177801 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f4003df0 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 10 client.15252 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56464bf5cb10 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f64ea7fc700 1 -- 172.21.15.64:0/3918177801 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56464c088220 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f64ea7fc700 1 -- 172.21.15.64:0/3918177801 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64f4044400 con 0x56464c1df4f0 2024-05-08T21:54:32.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 10 client.15252 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 20 client.15252 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c1f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 4 client.15252 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.200+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 --> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56464c1ae7c0 con 0x56464c1f39e0 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.201+0000 7f65027fc700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c1f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.201+0000 7f65027fc700 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c1f8c80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea495ae2c2b9ef6d server_cookie=9efd5b4a1af5f1aa in_seq=0 out_seq=0 2024-05-08T21:54:32.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.201+0000 7f64ea7fc700 10 client.15252 ms_handle_connect on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:32.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f64ea7fc700 1 -- 172.21.15.64:0/3918177801 <== mds.4 v2:172.21.15.160:6832/3542328594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56464c1ae7c0 con 0x56464c1f39e0 2024-05-08T21:54:32.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f64ea7fc700 10 client.15252 handle_command_reply: tid=0 2024-05-08T21:54:32.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 client.15252 shutdown 2024-05-08T21:54:32.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 msgr2=0x56464c1e8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1e8cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009d10 tx=0x7f64f4004220 comp rx=0 tx=0).stop 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 shutdown_connections 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c1f8c80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f64ea7fc700 0 client.15252 ms_handle_reset on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1e8cc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 --2- 172.21.15.64:0/3918177801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 >> 172.21.15.64:0/3918177801 conn(0x56464c13d1c0 msgr2=0x56464c1d9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 shutdown_connections 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 1 -- 172.21.15.64:0/3918177801 wait complete. 2024-05-08T21:54:32.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.202+0000 7f6515978b80 20 client.15252 trim_cache size 0 max 16384 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 Processor -- start 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 -- start start 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1df4f0 0x56464c1f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 0x56464c1e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c1dab80 con 0x56464c1e0810 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c1f5230 con 0x56464c1df4f0 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f6515978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c1f13c0 con 0x56464c1dfe50 2024-05-08T21:54:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.220+0000 7f65037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 0x56464c1e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 0x56464c1e8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39204/0 (socket says 172.21.15.64:39204) 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 -- 172.21.15.64:0/3148898723 learned_addr learned my addr 172.21.15.64:0/3148898723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65027fc700 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 -- 172.21.15.64:0/3148898723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 msgr2=0x56464c1f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f6502ffd700 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1df4f0 0x56464c1f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 -- 172.21.15.64:0/3148898723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1df4f0 msgr2=0x56464c1f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1df4f0 0x56464c1f08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 -- 172.21.15.64:0/3148898723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464c0f5510 con 0x56464c1e0810 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65027fc700 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f65037fe700 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 0x56464c1e8a70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64fc095f30 tx=0x7f64fc004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd72a0cc6b6ef6b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f64f9ffb700 1 -- 172.21.15.64:0/3148898723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc09d0d0 con 0x56464c1e0810 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f64f9ffb700 1 -- 172.21.15.64:0/3148898723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64fc09d2f0 con 0x56464c1e0810 2024-05-08T21:54:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.221+0000 7f64f9ffb700 1 -- 172.21.15.64:0/3148898723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0840b0 con 0x56464c1e0810 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 -- 172.21.15.64:0/3148898723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 msgr2=0x56464c1e8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 0x56464c1e8a70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64fc095f30 tx=0x7f64fc004f10 comp rx=0 tx=0).stop 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 -- 172.21.15.64:0/3148898723 shutdown_connections 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1df4f0 0x56464c1f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1e0810 0x56464c1e8a70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 --2- 172.21.15.64:0/3148898723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 -- 172.21.15.64:0/3148898723 >> 172.21.15.64:0/3148898723 conn(0x56464c13d1c0 msgr2=0x56464c1e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 -- 172.21.15.64:0/3148898723 shutdown_connections 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 -- 172.21.15.64:0/3148898723 wait complete. 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 -- start start 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.222+0000 7f6515978b80 1 Processor -- start 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c1eff30 con 0x56464c1df4f0 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c1f00a0 con 0x56464c1e0810 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c1f0260 con 0x56464c1dfe50 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 10 client.0 _ll_register_callbacks cb 0x56464c0d4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1ea480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39218/0 (socket says 172.21.15.64:39218) 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 -- 172.21.15.64:0/181848537 learned_addr learned my addr 172.21.15.64:0/181848537 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f65027fc700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1ea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f65037fe700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 -- 172.21.15.64:0/181848537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 msgr2=0x56464c1ea9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1ea9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 -- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 msgr2=0x56464c1ef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1ef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 -- 172.21.15.64:0/181848537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650419f3f0 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f65037fe700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6502ffd700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1ea480 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f64f4044710 tx=0x7f64f4044930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4cf810fb3326de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f64ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f64ea7fc700 1 -- 172.21.15.64:0/181848537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4044c60 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f64ea7fc700 1 -- 172.21.15.64:0/181848537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64f4040030 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64f401e040 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f64ea7fc700 1 -- 172.21.15.64:0/181848537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f4044dc0 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 10 client.15258 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.223+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56464bfb4c80 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f64ea7fc700 1 -- 172.21.15.64:0/181848537 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f401e040 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f64ea7fc700 1 -- 172.21.15.64:0/181848537 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64f40437e0 con 0x56464c1df4f0 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f6515978b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f6515978b80 10 client.15258 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-08T21:54:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f6515978b80 20 client.15258 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f6515978b80 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c24b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f6515978b80 4 client.15258 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 --> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56464c127e70 con 0x56464c1f39e0 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f65027fc700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c24b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f64ea7fc700 10 client.15258 ms_handle_connect on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.224+0000 7f65027fc700 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c24b550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2124168b0c4226 server_cookie=bafef67e85d8c911 in_seq=0 out_seq=0 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f64ea7fc700 1 -- 172.21.15.64:0/181848537 <== mds.4 v2:172.21.15.160:6832/3542328594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56464c127e70 con 0x56464c1f39e0 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f64ea7fc700 10 client.15258 handle_command_reply: tid=0 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 client.15258 shutdown 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 msgr2=0x56464c1ea480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1ea480 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f64f4044710 tx=0x7f64f4044930 comp rx=0 tx=0).stop 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 shutdown_connections 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:6832/3542328594,v1:172.21.15.160:6833/3542328594] conn(0x56464c1f39e0 0x56464c24b550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464c1e0810 0x56464c1ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56464c1df4f0 0x56464c1ea480 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f64ea7fc700 0 client.15258 ms_handle_reset on v2:172.21.15.160:6832/3542328594 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 --2- 172.21.15.64:0/181848537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56464c1dfe50 0x56464c1ea9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 >> 172.21.15.64:0/181848537 conn(0x56464c13d1c0 msgr2=0x56464c1e8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 shutdown_connections 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 1 -- 172.21.15.64:0/181848537 wait complete. 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f6515978b80 20 client.15258 trim_cache size 0 max 16384 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 4, 2024-05-08T21:54:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24422, 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 63.271850319999999, 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 81, 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 92.380421944000005 2024-05-08T21:54:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64f00a11b0 msgr2=0x7f64f00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64f00a11b0 0x7f64f00a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f64f4003820 tx=0x7f64f4005920 comp rx=0 tx=0).stop 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 msgr2=0x7f650419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 0x7f650419e440 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f64fc007ee0 tx=0x7f64fc00d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 shutdown_connections 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f64f00a11b0 0x7f64f00a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6504100f80 0x7f6504196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504102b10 0x7f650419e440 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 --2- 172.21.15.64:0/3391998658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6504102170 0x7f6504197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.225+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 >> 172.21.15.64:0/3391998658 conn(0x7f65040fc4f0 msgr2=0x7f65041055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.226+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 shutdown_connections 2024-05-08T21:54:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.226+0000 7f65097be700 1 -- 172.21.15.64:0/3391998658 wait complete. 2024-05-08T21:54:32.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-08T21:54:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2925752336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 msgr2=0x7f3ab8106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2925752336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab8106700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009990 tx=0x7f3aac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2925752336 shutdown_connections 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2925752336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 0x7f3ab8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2925752336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 0x7f3ab81090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2925752336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab8106700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2925752336 >> 172.21.15.64:0/2925752336 conn(0x7f3ab80fb3f0 msgr2=0x7f3ab80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2925752336 shutdown_connections 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2925752336 wait complete. 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 Processor -- start 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.365+0000 7f3abe6a8700 1 -- start start 2024-05-08T21:54:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 0x7f3ab8101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab80ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 0x7f3ab80ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8100670 con 0x7f3ab8104310 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab81007e0 con 0x7f3ab81019e0 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab81a0f70 con 0x7f3ab8106c40 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab80ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab80ff990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41244/0 (socket says 172.21.15.64:41244) 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 -- 172.21.15.64:0/2565153270 learned_addr learned my addr 172.21.15.64:0/2565153270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 0x7f3ab8101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 0x7f3ab80ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 -- 172.21.15.64:0/2565153270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 msgr2=0x7f3ab80ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 0x7f3ab80ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 -- 172.21.15.64:0/2565153270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 msgr2=0x7f3ab8101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 0x7f3ab8101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 -- 172.21.15.64:0/2565153270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0095f0 con 0x7f3ab8104310 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 0x7f3ab8101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 0x7f3ab80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab80ff990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0098e0 tx=0x7f3aac00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab57fa700 1 -- 172.21.15.64:0/2565153270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac01d070 con 0x7f3ab8104310 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab81a11a0 con 0x7f3ab8104310 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab57fa700 1 -- 172.21.15.64:0/2565153270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac005510 con 0x7f3ab8104310 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3ab57fa700 1 -- 172.21.15.64:0/2565153270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004c40 con 0x7f3ab8104310 2024-05-08T21:54:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.366+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81a1560 con 0x7f3ab8104310 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.369+0000 7f3ab57fa700 1 -- 172.21.15.64:0/2565153270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3aac00fbd0 con 0x7f3ab8104310 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.369+0000 7f3ab57fa700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3aa00a11b0 0x7f3aa00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.369+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3aa00a11b0 0x7f3aa00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.369+0000 7f3ab57fa700 1 -- 172.21.15.64:0/2565153270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3aac0cdb70 con 0x7f3ab8104310 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3aa00a11b0 0x7f3aa00a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8005d90 tx=0x7f3aa8005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 Processor -- start 2024-05-08T21:54:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 -- start start 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a6537100 con 0x5623a65e1810 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a6534ac0 con 0x5623a65e0e50 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3aca862b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a65303b0 con 0x5623a65e04f0 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65e1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41246/0 (socket says 172.21.15.64:41246) 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 -- 172.21.15.64:0/3086193309 learned_addr learned my addr 172.21.15.64:0/3086193309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 -- 172.21.15.64:0/3086193309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 msgr2=0x5623a65e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 -- 172.21.15.64:0/3086193309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 msgr2=0x5623a65e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.370+0000 7f3abcc45700 1 -- 172.21.15.64:0/3086193309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623a63b1360 con 0x5623a65e1810 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3abcc45700 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65e1c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009ca0 tx=0x7f3ab0007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c778b5d470cd3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3a9e7fc700 1 -- 172.21.15.64:0/3086193309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0004bb0 con 0x5623a65e1810 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3a9e7fc700 1 -- 172.21.15.64:0/3086193309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab0004d10 con 0x5623a65e1810 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3a9e7fc700 1 -- 172.21.15.64:0/3086193309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0014660 con 0x5623a65e1810 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3aca862b80 1 -- 172.21.15.64:0/3086193309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 msgr2=0x5623a65e1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.371+0000 7f3aca862b80 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65e1c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009ca0 tx=0x7f3ab0007500 comp rx=0 tx=0).stop 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- 172.21.15.64:0/3086193309 shutdown_connections 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65e1c90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 --2- 172.21.15.64:0/3086193309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- 172.21.15.64:0/3086193309 >> 172.21.15.64:0/3086193309 conn(0x5623a653e1c0 msgr2=0x5623a65dad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- 172.21.15.64:0/3086193309 shutdown_connections 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- 172.21.15.64:0/3086193309 wait complete. 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- start start 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 Processor -- start 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e04f0 0x5623a65e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e0e50 0x5623a65ea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a64cc510 con 0x5623a65e1810 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a64cc1b0 con 0x5623a65e04f0 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.372+0000 7f3aca862b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a63b3650 con 0x5623a65e0e50 2024-05-08T21:54:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3aca862b80 10 client.0 _ll_register_callbacks cb 0x5623a64d5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41258/0 (socket says 172.21.15.64:41258) 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 -- 172.21.15.64:0/2773351009 learned_addr learned my addr 172.21.15.64:0/2773351009 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e0e50 0x5623a65ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e04f0 0x5623a65e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 -- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e0e50 msgr2=0x5623a65ea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e0e50 0x5623a65ea310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 -- 172.21.15.64:0/2773351009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e04f0 msgr2=0x5623a65e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e04f0 0x5623a65e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 -- 172.21.15.64:0/2773351009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab0009900 con 0x5623a65e1810 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e04f0 0x5623a65e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f1350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00098a0 tx=0x7f3ab00055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c96060fa8786003 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3a977fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3a977fe700 1 -- 172.21.15.64:0/2773351009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0003a00 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3a977fe700 1 -- 172.21.15.64:0/2773351009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3ab0003b60 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623a6489220 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.373+0000 7f3a977fe700 1 -- 172.21.15.64:0/2773351009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab0003d20 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 10 client.15270 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5623a635db10 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3a977fe700 1 -- 172.21.15.64:0/2773351009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab002e020 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3a977fe700 1 -- 172.21.15.64:0/2773351009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ab001c040 con 0x5623a65e1810 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 10 client.15270 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:54:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 20 client.15270 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a65f4ec0 0x5623a65fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 4 client.15270 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 --> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623a65af7c0 con 0x5623a65f4ec0 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.374+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a65f4ec0 0x5623a65fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.375+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a65f4ec0 0x5623a65fa1e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd8dea06a9caaec6 server_cookie=fa1d1762af643e4f in_seq=0 out_seq=0 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.375+0000 7f3a977fe700 10 client.15270 ms_handle_connect on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.375+0000 7f3a977fe700 1 -- 172.21.15.64:0/2773351009 <== mds.3 v2:172.21.15.186:6832/1716244539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623a65af7c0 con 0x5623a65f4ec0 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.375+0000 7f3a977fe700 10 client.15270 handle_command_reply: tid=0 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.375+0000 7f3aca862b80 1 client.15270 shutdown 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 msgr2=0x5623a65f1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f1350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00098a0 tx=0x7f3ab00055d0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 shutdown_connections 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a65f4ec0 0x5623a65fa1e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e04f0 0x5623a65e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f1350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3a977fe700 0 client.15270 ms_handle_reset on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2773351009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e0e50 0x5623a65ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 >> 172.21.15.64:0/2773351009 conn(0x5623a653e1c0 msgr2=0x5623a65df2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 shutdown_connections 2024-05-08T21:54:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 1 -- 172.21.15.64:0/2773351009 wait complete. 2024-05-08T21:54:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.376+0000 7f3aca862b80 20 client.15270 trim_cache size 0 max 16384 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 Processor -- start 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 -- start start 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 0x5623a65e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e1810 0x5623a65f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a6529150 con 0x5623a65e0e50 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a652d010 con 0x5623a65e1810 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3aca862b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a64cf5a0 con 0x5623a65e04f0 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 0x5623a65e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 0x5623a65e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41262/0 (socket says 172.21.15.64:41262) 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 -- 172.21.15.64:0/818804739 learned_addr learned my addr 172.21.15.64:0/818804739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 -- 172.21.15.64:0/818804739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 msgr2=0x5623a65e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 -- 172.21.15.64:0/818804739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e1810 msgr2=0x5623a65f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e1810 0x5623a65f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 -- 172.21.15.64:0/818804739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623a64f6510 con 0x5623a65e0e50 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 0x5623a65e9350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3aac09ae90 tx=0x7f3aac009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172fcba1a9051c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.395+0000 7f3a9effd700 1 -- 172.21.15.64:0/818804739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac09af00 con 0x5623a65e0e50 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3a9effd700 1 -- 172.21.15.64:0/818804739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac09ed30 con 0x5623a65e0e50 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3a9effd700 1 -- 172.21.15.64:0/818804739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0a2e60 con 0x5623a65e0e50 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 -- 172.21.15.64:0/818804739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 msgr2=0x5623a65e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 0x5623a65e9350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3aac09ae90 tx=0x7f3aac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 -- 172.21.15.64:0/818804739 shutdown_connections 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e1810 0x5623a65f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 --2- 172.21.15.64:0/818804739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e0e50 0x5623a65e9350 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 -- 172.21.15.64:0/818804739 >> 172.21.15.64:0/818804739 conn(0x5623a653e1c0 msgr2=0x5623a65e1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 -- 172.21.15.64:0/818804739 shutdown_connections 2024-05-08T21:54:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 -- 172.21.15.64:0/818804739 wait complete. 2024-05-08T21:54:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.396+0000 7f3aca862b80 1 -- start start 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 Processor -- start 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a65f1270 con 0x5623a65e1810 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a65f13e0 con 0x5623a65e0e50 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a65f15a0 con 0x5623a65e04f0 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3aca862b80 10 client.0 _ll_register_callbacks cb 0x5623a64d5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3abcc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3abcc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41274/0 (socket says 172.21.15.64:41274) 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3abcc45700 1 -- 172.21.15.64:0/2251508999 learned_addr learned my addr 172.21.15.64:0/2251508999 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3ab77fe700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.397+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3abcc45700 1 -- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 msgr2=0x5623a65ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3abcc45700 1 -- 172.21.15.64:0/2251508999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 msgr2=0x5623a65f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3abcc45700 1 -- 172.21.15.64:0/2251508999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81a1560 con 0x5623a65e1810 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3abcc45700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f0d30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0008fb0 tx=0x7f3ab0007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1284fe7468a77fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3a977fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3a977fe700 1 -- 172.21.15.64:0/2251508999 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00293e0 con 0x5623a65e1810 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ab0004d10 con 0x5623a65e1810 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3a977fe700 1 -- 172.21.15.64:0/2251508999 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3ab002f080 con 0x5623a65e1810 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3a977fe700 1 -- 172.21.15.64:0/2251508999 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab002a770 con 0x5623a65e1810 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 10 client.15276 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5623a63b5c80 con 0x5623a65e1810 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3a977fe700 1 -- 172.21.15.64:0/2251508999 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab0004d10 con 0x5623a65e1810 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3a977fe700 1 -- 172.21.15.64:0/2251508999 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ab001c070 con 0x5623a65e1810 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 10 client.15276 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 20 client.15276 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a6650720 0x5623a664d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 4 client.15276 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.398+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 --> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623a6528e70 con 0x5623a6650720 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.399+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a6650720 0x5623a664d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.399+0000 7f3ab7fff700 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a6650720 0x5623a664d0d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66b4e061bf1c7247 server_cookie=3713467948a3476a in_seq=0 out_seq=0 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.399+0000 7f3a977fe700 10 client.15276 ms_handle_connect on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3a977fe700 1 -- 172.21.15.64:0/2251508999 <== mds.3 v2:172.21.15.186:6832/1716244539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5623a6528e70 con 0x5623a6650720 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3a977fe700 10 client.15276 handle_command_reply: tid=0 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 client.15276 shutdown 2024-05-08T21:54:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 msgr2=0x5623a65f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f0d30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0008fb0 tx=0x7f3ab0007900 comp rx=0 tx=0).stop 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 shutdown_connections 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:6832/1716244539,v1:172.21.15.186:6833/1716244539] conn(0x5623a6650720 0x5623a664d0d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a65e1810 0x5623a65f0d30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623a65e04f0 0x5623a65ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3a977fe700 0 client.15276 ms_handle_reset on v2:172.21.15.186:6832/1716244539 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 --2- 172.21.15.64:0/2251508999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623a65e0e50 0x5623a65f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 >> 172.21.15.64:0/2251508999 conn(0x5623a653e1c0 msgr2=0x5623a65e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 shutdown_connections 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 1 -- 172.21.15.64:0/2251508999 wait complete. 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3aca862b80 20 client.15276 trim_cache size 0 max 16384 2024-05-08T21:54:32.406 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 3, 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24427, 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 64.456464812999997, 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 81, 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 94.336834852999999 2024-05-08T21:54:32.407 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3aa00a11b0 msgr2=0x7f3aa00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3aa00a11b0 0x7f3aa00a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8005d90 tx=0x7f3aa8005d20 comp rx=0 tx=0).stop 2024-05-08T21:54:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 msgr2=0x7f3ab80ff990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.400+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab80ff990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0098e0 tx=0x7f3aac00f720 comp rx=0 tx=0).stop 2024-05-08T21:54:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 shutdown_connections 2024-05-08T21:54:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3aa00a11b0 0x7f3aa00a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ab81019e0 0x7f3ab8101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ab8106c40 0x7f3ab80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 --2- 172.21.15.64:0/2565153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8104310 0x7f3ab80ff990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 >> 172.21.15.64:0/2565153270 conn(0x7f3ab80fb3f0 msgr2=0x7f3ab81023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 shutdown_connections 2024-05-08T21:54:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.401+0000 7f3abe6a8700 1 -- 172.21.15.64:0/2565153270 wait complete. 2024-05-08T21:54:32.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T21:54:32.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 -- 172.21.15.64:0/2694577041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 msgr2=0x7f42a01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 --2- 172.21.15.64:0/2694577041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01025d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f429c0099c0 tx=0x7f429c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 -- 172.21.15.64:0/2694577041 shutdown_connections 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 --2- 172.21.15.64:0/2694577041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0102b10 0x7f42a01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 --2- 172.21.15.64:0/2694577041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42a0100f80 0x7f42a01013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 --2- 172.21.15.64:0/2694577041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01025d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 -- 172.21.15.64:0/2694577041 >> 172.21.15.64:0/2694577041 conn(0x7f42a00fc4f0 msgr2=0x7f42a00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 -- 172.21.15.64:0/2694577041 shutdown_connections 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 -- 172.21.15.64:0/2694577041 wait complete. 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 Processor -- start 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.540+0000 7f42a8083700 1 -- start start 2024-05-08T21:54:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0100f80 0x7f42a0196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42a0102b10 0x7f42a019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a019e9d0 con 0x7f42a0102170 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a019eb10 con 0x7f42a0100f80 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a019eca0 con 0x7f42a0102b10 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41290/0 (socket says 172.21.15.64:41290) 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 -- 172.21.15.64:0/2002497837 learned_addr learned my addr 172.21.15.64:0/2002497837 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0100f80 0x7f42a0196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a6620700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42a0102b10 0x7f42a019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 -- 172.21.15.64:0/2002497837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42a0102b10 msgr2=0x7f42a019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42a0102b10 0x7f42a019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 -- 172.21.15.64:0/2002497837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0100f80 msgr2=0x7f42a0196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0100f80 0x7f42a0196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 -- 172.21.15.64:0/2002497837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c0095f0 con 0x7f42a0102170 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0100f80 0x7f42a0196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a561e700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01973e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f429c003730 tx=0x7f429c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f4296ffd700 1 -- 172.21.15.64:0/2002497837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c01d070 con 0x7f42a0102170 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f4296ffd700 1 -- 172.21.15.64:0/2002497837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f429c0059a0 con 0x7f42a0102170 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a019ef20 con 0x7f42a0102170 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.541+0000 7f4296ffd700 1 -- 172.21.15.64:0/2002497837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c004a70 con 0x7f42a0102170 2024-05-08T21:54:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.542+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a019f340 con 0x7f42a0102170 2024-05-08T21:54:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.544+0000 7f4296ffd700 1 -- 172.21.15.64:0/2002497837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f429c00fb60 con 0x7f42a0102170 2024-05-08T21:54:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.544+0000 7f4296ffd700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f428c0a11b0 0x7f428c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.544+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f428c0a11b0 0x7f428c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.544+0000 7f4296ffd700 1 -- 172.21.15.64:0/2002497837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f429c0cd6a0 con 0x7f42a0102170 2024-05-08T21:54:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.545+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f428c0a11b0 0x7f428c0a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42a0101fd0 tx=0x7f4290009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 Processor -- start 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 -- start start 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf9b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9bc20 0x55ffddf9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddeed100 con 0x55ffddf9a900 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddeeaac0 con 0x55ffddf9bc20 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42b423db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddee63b0 con 0x55ffddf9b260 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41304/0 (socket says 172.21.15.64:41304) 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3644274203 learned_addr learned my addr 172.21.15.64:0/3644274203 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a561e700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf9b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a6620700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9bc20 0x55ffddf9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3644274203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 msgr2=0x55ffddf9b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf9b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3644274203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9bc20 msgr2=0x55ffddf9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9bc20 0x55ffddf9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3644274203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffddd67360 con 0x55ffddf9a900 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a6620700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9bc20 0x55ffddf9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.546+0000 7f42a561e700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf9b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.547+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ad20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4290011ed0 tx=0x7f4290014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c279c9fa8a4ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.547+0000 7f428bfff700 1 -- 172.21.15.64:0/3644274203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429001f620 con 0x55ffddf9a900 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.547+0000 7f428bfff700 1 -- 172.21.15.64:0/3644274203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4290016070 con 0x55ffddf9a900 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.547+0000 7f428bfff700 1 -- 172.21.15.64:0/3644274203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429001da70 con 0x55ffddf9a900 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.547+0000 7f42b423db80 1 -- 172.21.15.64:0/3644274203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 msgr2=0x55ffddf9ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.547+0000 7f42b423db80 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ad20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4290011ed0 tx=0x7f4290014000 comp rx=0 tx=0).stop 2024-05-08T21:54:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- 172.21.15.64:0/3644274203 shutdown_connections 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9bc20 0x55ffddf9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ad20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 --2- 172.21.15.64:0/3644274203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf9b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- 172.21.15.64:0/3644274203 >> 172.21.15.64:0/3644274203 conn(0x55ffddef41c0 msgr2=0x55ffddf90d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- 172.21.15.64:0/3644274203 shutdown_connections 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- 172.21.15.64:0/3644274203 wait complete. 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- start start 2024-05-08T21:54:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 Processor -- start 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9a900 0x55ffddfa4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfa46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdde82510 con 0x55ffddf9bc20 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdde821b0 con 0x55ffddf9b260 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddd69650 con 0x55ffddf9a900 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.548+0000 7f42b423db80 10 client.0 _ll_register_callbacks cb 0x55ffdde8bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41310/0 (socket says 172.21.15.64:41310) 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 -- 172.21.15.64:0/3531118450 learned_addr learned my addr 172.21.15.64:0/3531118450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9a900 0x55ffddfa4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a561e700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfa46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 -- 172.21.15.64:0/3531118450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9a900 msgr2=0x55ffddfa4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9a900 0x55ffddfa4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 -- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 msgr2=0x55ffddfa46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfa46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 -- 172.21.15.64:0/3531118450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4290011c60 con 0x55ffddf9bc20 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a561e700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfa46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42a6620700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfab710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4298007be0 tx=0x7f4298007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785991349f33dc04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42897fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42897fa700 1 -- 172.21.15.64:0/3531118450 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800e040 con 0x55ffddf9bc20 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42897fa700 1 -- 172.21.15.64:0/3531118450 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4298004bb0 con 0x55ffddf9bc20 2024-05-08T21:54:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffdde3f220 con 0x55ffddf9bc20 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42897fa700 1 -- 172.21.15.64:0/3531118450 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4298004d70 con 0x55ffddf9bc20 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42b423db80 10 client.15288 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.549+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ffddd13b10 con 0x55ffddf9bc20 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42897fa700 1 -- 172.21.15.64:0/3531118450 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffdde3f220 con 0x55ffddf9bc20 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42897fa700 1 -- 172.21.15.64:0/3531118450 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f42980169a0 con 0x55ffddf9bc20 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42b423db80 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42b423db80 10 client.15288 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42b423db80 20 client.15288 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42b423db80 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffddfae910 0x55ffddfb4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42b423db80 4 client.15288 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 --> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffddf657c0 con 0x55ffddfae910 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.550+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffddfae910 0x55ffddfb4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffddfae910 0x55ffddfb4730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5bd54cd4b222174 server_cookie=7bdd5f6095cdfc57 in_seq=0 out_seq=0 2024-05-08T21:54:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42897fa700 10 client.15288 ms_handle_connect on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42897fa700 1 -- 172.21.15.64:0/3531118450 <== mds.2 v2:172.21.15.160:6836/4233185014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffddf657c0 con 0x55ffddfae910 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42897fa700 10 client.15288 handle_command_reply: tid=0 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 client.15288 shutdown 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 msgr2=0x55ffddfab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfab710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4298007be0 tx=0x7f4298007c10 comp rx=0 tx=0).stop 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 shutdown_connections 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffddfae910 0x55ffddfb4730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfab710 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42897fa700 0 client.15288 ms_handle_reset on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9a900 0x55ffddfa4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 --2- 172.21.15.64:0/3531118450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfa46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.551+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 >> 172.21.15.64:0/3531118450 conn(0x55ffddef41c0 msgr2=0x55ffddf90550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.552+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 shutdown_connections 2024-05-08T21:54:32.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.552+0000 7f42b423db80 1 -- 172.21.15.64:0/3531118450 wait complete. 2024-05-08T21:54:32.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.552+0000 7f42b423db80 20 client.15288 trim_cache size 0 max 16384 2024-05-08T21:54:32.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 Processor -- start 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 -- start start 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9bc20 0x55ffddfa2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddf657c0 con 0x55ffddf9a900 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdde3f220 con 0x55ffddf9b260 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42b423db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddee3010 con 0x55ffddf9bc20 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41318/0 (socket says 172.21.15.64:41318) 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3312712188 learned_addr learned my addr 172.21.15.64:0/3312712188 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3312712188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9bc20 msgr2=0x55ffddfa2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a6620700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9bc20 0x55ffddfa2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a561e700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9bc20 0x55ffddfa2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3312712188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 msgr2=0x55ffddfab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 -- 172.21.15.64:0/3312712188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffddeac510 con 0x55ffddf9a900 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a6620700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9bc20 0x55ffddfa2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a561e700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.570+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ace0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f429001d5a0 tx=0x7f429002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4e46fe1af8369d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f4294ff9700 1 -- 172.21.15.64:0/3312712188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429002e040 con 0x55ffddf9a900 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f4294ff9700 1 -- 172.21.15.64:0/3312712188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f429002f040 con 0x55ffddf9a900 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f4294ff9700 1 -- 172.21.15.64:0/3312712188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290036350 con 0x55ffddf9a900 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 -- 172.21.15.64:0/3312712188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 msgr2=0x55ffddf9ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ace0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f429001d5a0 tx=0x7f429002b040 comp rx=0 tx=0).stop 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 -- 172.21.15.64:0/3312712188 shutdown_connections 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9bc20 0x55ffddfa2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9a900 0x55ffddf9ace0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 --2- 172.21.15.64:0/3312712188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9b260 0x55ffddfab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 -- 172.21.15.64:0/3312712188 >> 172.21.15.64:0/3312712188 conn(0x55ffddef41c0 msgr2=0x55ffddf9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 -- 172.21.15.64:0/3312712188 shutdown_connections 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.571+0000 7f42b423db80 1 -- 172.21.15.64:0/3312712188 wait complete. 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 -- start start 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 Processor -- start 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9a900 0x55ffddf8ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfaac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddfab1c0 con 0x55ffddf9bc20 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddfab330 con 0x55ffddf9a900 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffddfab4a0 con 0x55ffddf9b260 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42b423db80 10 client.0 _ll_register_callbacks cb 0x55ffdde8bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42a6620700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfaac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42a6620700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfaac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41320/0 (socket says 172.21.15.64:41320) 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42a6620700 1 -- 172.21.15.64:0/3381597361 learned_addr learned my addr 172.21.15.64:0/3381597361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9a900 0x55ffddf8ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.572+0000 7f42a561e700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42a6620700 1 -- 172.21.15.64:0/3381597361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 msgr2=0x55ffddf90450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42a6620700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf90450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42a6620700 1 -- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9a900 msgr2=0x55ffddf8ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42a6620700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9a900 0x55ffddf8ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42a6620700 1 -- 172.21.15.64:0/3381597361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffddd67360 con 0x55ffddf9bc20 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42a6620700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfaac80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4298016d00 tx=0x7f42980060b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df49c000c89578d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42897fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42897fa700 1 -- 172.21.15.64:0/3381597361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800e040 con 0x55ffddf9bc20 2024-05-08T21:54:32.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42897fa700 1 -- 172.21.15.64:0/3381597361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4298004c80 con 0x55ffddf9bc20 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4290016070 con 0x55ffddf9bc20 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42897fa700 1 -- 172.21.15.64:0/3381597361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4298016ea0 con 0x55ffddf9bc20 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 10 client.15294 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ffddd6bc80 con 0x55ffddf9bc20 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42897fa700 1 -- 172.21.15.64:0/3381597361 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4290016070 con 0x55ffddf9bc20 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42897fa700 1 -- 172.21.15.64:0/3381597361 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f42980168b0 con 0x55ffddf9bc20 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 10 client.15294 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 20 client.15294 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffde00aa70 0x55ffde0073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 4 client.15294 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.573+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 --> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ffddedee70 con 0x55ffde00aa70 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffde00aa70 0x55ffde0073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42a5e1f700 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffde00aa70 0x55ffde0073f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a9a8a9bc59b48a1 server_cookie=1874faa3ee12ba25 in_seq=0 out_seq=0 2024-05-08T21:54:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42897fa700 10 client.15294 ms_handle_connect on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42897fa700 1 -- 172.21.15.64:0/3381597361 <== mds.2 v2:172.21.15.160:6836/4233185014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ffddedee70 con 0x55ffde00aa70 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42897fa700 10 client.15294 handle_command_reply: tid=0 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42b423db80 1 client.15294 shutdown 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 msgr2=0x55ffddfaac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42b423db80 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfaac80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4298016d00 tx=0x7f42980060b0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 shutdown_connections 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.574+0000 7f42b423db80 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:6836/4233185014,v1:172.21.15.160:6837/4233185014] conn(0x55ffde00aa70 0x55ffde0073f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffddf9a900 0x55ffddf8ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42897fa700 0 client.15294 ms_handle_reset on v2:172.21.15.160:6836/4233185014 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffddf9bc20 0x55ffddfaac80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 1 --2- 172.21.15.64:0/3381597361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffddf9b260 0x55ffddf90450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 >> 172.21.15.64:0/3381597361 conn(0x55ffddef41c0 msgr2=0x55ffddfa4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 shutdown_connections 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 1 -- 172.21.15.64:0/3381597361 wait complete. 2024-05-08T21:54:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42b423db80 20 client.15294 trim_cache size 0 max 16384 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24476, 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 64.983213043000006, 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 81, 2024-05-08T21:54:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 82.239285494000001 2024-05-08T21:54:32.583 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:32.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f428c0a11b0 msgr2=0x7f428c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f428c0a11b0 0x7f428c0a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42a0101fd0 tx=0x7f4290009260 comp rx=0 tx=0).stop 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 msgr2=0x7f42a01973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01973e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f429c003730 tx=0x7f429c00f670 comp rx=0 tx=0).stop 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 shutdown_connections 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f428c0a11b0 0x7f428c0a3670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a0100f80 0x7f42a0196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42a0102b10 0x7f42a019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 --2- 172.21.15.64:0/2002497837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a0102170 0x7f42a01973e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 >> 172.21.15.64:0/2002497837 conn(0x7f42a00fc4f0 msgr2=0x7f42a0105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 shutdown_connections 2024-05-08T21:54:32.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.575+0000 7f42a8083700 1 -- 172.21.15.64:0/2002497837 wait complete. 2024-05-08T21:54:32.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T21:54:32.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 -- 172.21.15.64:0/2201097967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c100fc0 msgr2=0x7f9a2c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 --2- 172.21.15.64:0/2201097967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c100fc0 0x7f9a2c101440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009990 tx=0x7f9a14009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 -- 172.21.15.64:0/2201097967 shutdown_connections 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 --2- 172.21.15.64:0/2201097967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 0x7f9a2c1082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 --2- 172.21.15.64:0/2201097967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 --2- 172.21.15.64:0/2201097967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c100fc0 0x7f9a2c101440 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 -- 172.21.15.64:0/2201097967 >> 172.21.15.64:0/2201097967 conn(0x7f9a2c0fb400 msgr2=0x7f9a2c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 -- 172.21.15.64:0/2201097967 shutdown_connections 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 -- 172.21.15.64:0/2201097967 wait complete. 2024-05-08T21:54:32.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 Processor -- start 2024-05-08T21:54:32.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.715+0000 7f9a31590700 1 -- start start 2024-05-08T21:54:32.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c100fc0 0x7f9a2c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 0x7f9a2c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c19ea10 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c19eb50 con 0x7f9a2c101980 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c19ece0 con 0x7f9a2c100fc0 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41346/0 (socket says 172.21.15.64:41346) 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 -- 172.21.15.64:0/2729562633 learned_addr learned my addr 172.21.15.64:0/2729562633 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c100fc0 0x7f9a2c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 0x7f9a2c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 -- 172.21.15.64:0/2729562633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c100fc0 msgr2=0x7f9a2c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c100fc0 0x7f9a2c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 -- 172.21.15.64:0/2729562633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 msgr2=0x7f9a2c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 0x7f9a2c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 -- 172.21.15.64:0/2729562633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140095f0 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 0x7f9a2c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a2affd700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c196ee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00d950 tx=0x7f9a1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a23fff700 1 -- 172.21.15.64:0/2729562633 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c009940 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a23fff700 1 -- 172.21.15.64:0/2729562633 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c010460 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c19efc0 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a23fff700 1 -- 172.21.15.64:0/2729562633 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00f620 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.716+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c19f3e0 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.719+0000 7f9a23fff700 1 -- 172.21.15.64:0/2729562633 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9a1c0108d0 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.719+0000 7f9a23fff700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9a180a11b0 0x7f9a180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.720+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9a180a11b0 0x7f9a180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.720+0000 7f9a23fff700 1 -- 172.21.15.64:0/2729562633 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9a1c0cc8b0 con 0x7f9a2c0ffe70 2024-05-08T21:54:32.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.720+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9a180a11b0 0x7f9a180a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400b5c0 tx=0x7f9a14003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 Processor -- start 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 -- start start 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a45910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a46c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76499c100 con 0x55f764a454f0 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764999ac0 con 0x55f764a46810 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7649953b0 con 0x55f764a45e50 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.721+0000 7f9a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a45910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a45910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41350/0 (socket says 172.21.15.64:41350) 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 -- 172.21.15.64:0/1072272857 learned_addr learned my addr 172.21.15.64:0/1072272857 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a46c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 -- 172.21.15.64:0/1072272857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 msgr2=0x55f764a462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 -- 172.21.15.64:0/1072272857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 msgr2=0x55f764a46c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a46c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 -- 172.21.15.64:0/1072272857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f764816360 con 0x55f764a454f0 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a46c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a2affd700 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a45910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c096620 tx=0x7f9a1c099610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c06b7c529fb9e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a217fa700 1 -- 172.21.15.64:0/1072272857 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0a2030 con 0x55f764a454f0 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a217fa700 1 -- 172.21.15.64:0/1072272857 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c0ca4c0 con 0x55f764a454f0 2024-05-08T21:54:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.722+0000 7f9a217fa700 1 -- 172.21.15.64:0/1072272857 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0992e0 con 0x55f764a454f0 2024-05-08T21:54:32.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1072272857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 msgr2=0x55f764a45910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a45910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c096620 tx=0x7f9a1c099610 comp rx=0 tx=0).stop 2024-05-08T21:54:32.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1072272857 shutdown_connections 2024-05-08T21:54:32.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a46c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a45910 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1072272857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1072272857 >> 172.21.15.64:0/1072272857 conn(0x55f7649a31c0 msgr2=0x55f764a3fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1072272857 shutdown_connections 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.723+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1072272857 wait complete. 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 -- start start 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 Processor -- start 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a53190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a5a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764931510 con 0x55f764a454f0 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7649311b0 con 0x55f764a46810 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764818650 con 0x55f764a45e50 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a3d74ab80 10 client.0 _ll_register_callbacks cb 0x55f76493aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a53190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a53190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41360/0 (socket says 172.21.15.64:41360) 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a2affd700 1 -- 172.21.15.64:0/3948939975 learned_addr learned my addr 172.21.15.64:0/3948939975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.724+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2affd700 1 -- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 msgr2=0x55f764a536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2affd700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2affd700 1 -- 172.21.15.64:0/3948939975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 msgr2=0x55f764a5a710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2affd700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a5a710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2affd700 1 -- 172.21.15.64:0/3948939975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c19f3e0 con 0x55f764a454f0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a2affd700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a53190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b2f0 tx=0x7f9a1c099d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b121975c642cd91f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.725+0000 7f9a127fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a127fc700 1 -- 172.21.15.64:0/3948939975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0a2030 con 0x55f764a454f0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a127fc700 1 -- 172.21.15.64:0/3948939975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9a1c0992b0 con 0x55f764a454f0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7648ee220 con 0x55f764a454f0 2024-05-08T21:54:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a127fc700 1 -- 172.21.15.64:0/3948939975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1c0a1ba0 con 0x55f764a454f0 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 10 client.15306 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f7647c2b10 con 0x55f764a454f0 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a127fc700 1 -- 172.21.15.64:0/3948939975 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7648ee220 con 0x55f764a454f0 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a127fc700 1 -- 172.21.15.64:0/3948939975 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9a1c095a30 con 0x55f764a454f0 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 10 client.15306 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 20 client.15306 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764a5df60 0x55f764a63240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 4 client.15306 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.726+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 --> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f764a147c0 con 0x55f764a5df60 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.727+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764a5df60 0x55f764a63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.727+0000 7f9a127fc700 10 client.15306 ms_handle_connect on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:32.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.727+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764a5df60 0x55f764a63240 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce1b5cfa948073d0 server_cookie=91d507c4aa902d6e in_seq=0 out_seq=0 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a127fc700 1 -- 172.21.15.64:0/3948939975 <== mds.1 v2:172.21.15.186:6838/539285459 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f764a147c0 con 0x55f764a5df60 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a127fc700 10 client.15306 handle_command_reply: tid=0 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 client.15306 shutdown 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 msgr2=0x55f764a53190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a53190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b2f0 tx=0x7f9a1c099d30 comp rx=0 tx=0).stop 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 shutdown_connections 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764a5df60 0x55f764a63240 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a46810 0x55f764a5a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a454f0 0x55f764a53190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/3948939975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a45e50 0x55f764a536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 >> 172.21.15.64:0/3948939975 conn(0x55f7649a31c0 msgr2=0x55f764a3f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a127fc700 0 client.15306 ms_handle_reset on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 shutdown_connections 2024-05-08T21:54:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/3948939975 wait complete. 2024-05-08T21:54:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.728+0000 7f9a3d74ab80 20 client.15306 trim_cache size 0 max 16384 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.746+0000 7f9a3d74ab80 1 Processor -- start 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.746+0000 7f9a3d74ab80 1 -- start start 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a5adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a5a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764a147c0 con 0x55f764a45e50 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7648ee220 con 0x55f764a454f0 2024-05-08T21:54:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7649532b0 con 0x55f764a46810 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a5adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a5adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41372/0 (socket says 172.21.15.64:41372) 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/2300937662 learned_addr learned my addr 172.21.15.64:0/2300937662 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2affd700 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a5a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/2300937662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 msgr2=0x55f764a5a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a5a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/2300937662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 msgr2=0x55f764a537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/2300937662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76495b510 con 0x55f764a45e50 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a5a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a5adc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009990 tx=0x7f9a14020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb0408259f6d5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.747+0000 7f9a21ffb700 1 -- 172.21.15.64:0/2300937662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140056c0 con 0x55f764a45e50 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a21ffb700 1 -- 172.21.15.64:0/2300937662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1401e040 con 0x55f764a45e50 2024-05-08T21:54:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a21ffb700 1 -- 172.21.15.64:0/2300937662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14003e40 con 0x55f764a45e50 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/2300937662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 msgr2=0x55f764a5adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a5adc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009990 tx=0x7f9a14020000 comp rx=0 tx=0).stop 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/2300937662 shutdown_connections 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a5a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/2300937662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a5adc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/2300937662 >> 172.21.15.64:0/2300937662 conn(0x55f7649a31c0 msgr2=0x55f764a46bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/2300937662 shutdown_connections 2024-05-08T21:54:32.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.748+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/2300937662 wait complete. 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 -- start start 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 Processor -- start 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a54de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764a5a1e0 con 0x55f764a45e50 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764a5a350 con 0x55f764a454f0 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764a5a510 con 0x55f764a46810 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a3d74ab80 10 client.0 _ll_register_callbacks cb 0x55f76493aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a54de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a54de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41388/0 (socket says 172.21.15.64:41388) 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/1958472705 learned_addr learned my addr 172.21.15.64:0/1958472705 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2affd700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 msgr2=0x55f764a59ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a59ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/1958472705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 msgr2=0x55f764a548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 -- 172.21.15.64:0/1958472705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c108bd0 con 0x55f764a45e50 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2b7fe700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a2a7fc700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a54de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c00 tx=0x7f9a140359f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb1ea6e8a66093b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.749+0000 7f9a127fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a127fc700 1 -- 172.21.15.64:0/1958472705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140352f0 con 0x55f764a45e50 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a127fc700 1 -- 172.21.15.64:0/1958472705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9a14035450 con 0x55f764a45e50 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a1c0ca4c0 con 0x55f764a45e50 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a127fc700 1 -- 172.21.15.64:0/1958472705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a14035610 con 0x55f764a45e50 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 10 client.15312 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f76481ac80 con 0x55f764a45e50 2024-05-08T21:54:32.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a127fc700 1 -- 172.21.15.64:0/1958472705 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c0ca4c0 con 0x55f764a45e50 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a127fc700 1 -- 172.21.15.64:0/1958472705 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9a14004870 con 0x55f764a45e50 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 20 client.15312 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764ab90f0 0x55f764ab5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 4 client.15312 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.750+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 --> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f76498de70 con 0x55f764ab90f0 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a2affd700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764ab90f0 0x55f764ab5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a127fc700 10 client.15312 ms_handle_connect on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a2affd700 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764ab90f0 0x55f764ab5ae0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c8c4e2f0c910585 server_cookie=bcf701d8055adf75 in_seq=0 out_seq=0 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a127fc700 1 -- 172.21.15.64:0/1958472705 <== mds.1 v2:172.21.15.186:6838/539285459 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f76498de70 con 0x55f764ab90f0 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a127fc700 10 client.15312 handle_command_reply: tid=0 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 client.15312 shutdown 2024-05-08T21:54:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 msgr2=0x55f764a54de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a54de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c00 tx=0x7f9a140359f0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 shutdown_connections 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:6838/539285459,v1:172.21.15.186:6839/539285459] conn(0x55f764ab90f0 0x55f764ab5ae0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f764a454f0 0x55f764a548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f764a46810 0x55f764a59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a127fc700 0 client.15312 ms_handle_reset on v2:172.21.15.186:6838/539285459 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 --2- 172.21.15.64:0/1958472705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f764a45e50 0x55f764a54de0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 >> 172.21.15.64:0/1958472705 conn(0x55f7649a31c0 msgr2=0x55f764a526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 shutdown_connections 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 1 -- 172.21.15.64:0/1958472705 wait complete. 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.751+0000 7f9a3d74ab80 20 client.15312 trim_cache size 0 max 16384 2024-05-08T21:54:32.758 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24484, 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 66.166950753999998, 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 81, 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 80.673986209000006 2024-05-08T21:54:32.759 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9a180a11b0 msgr2=0x7f9a180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9a180a11b0 0x7f9a180a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400b5c0 tx=0x7f9a14003790 comp rx=0 tx=0).stop 2024-05-08T21:54:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 msgr2=0x7f9a2c196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c196ee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00d950 tx=0x7f9a1c00dc60 comp rx=0 tx=0).stop 2024-05-08T21:54:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 shutdown_connections 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f9a180a11b0 0x7f9a180a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2c101980 0x7f9a2c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a2c0ffe70 0x7f9a2c196ee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 --2- 172.21.15.64:0/2729562633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c100fc0 0x7f9a2c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 >> 172.21.15.64:0/2729562633 conn(0x7f9a2c0fb400 msgr2=0x7f9a2c069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 shutdown_connections 2024-05-08T21:54:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.752+0000 7f9a31590700 1 -- 172.21.15.64:0/2729562633 wait complete. 2024-05-08T21:54:32.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T21:54:32.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 -- 172.21.15.64:0/3923620426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 msgr2=0x7fa698102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3923620426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa698102650 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009990 tx=0x7fa68c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 -- 172.21.15.64:0/3923620426 shutdown_connections 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3923620426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa698102b90 0x7fa698109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3923620426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6980ff220 0x7fa6980ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3923620426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa698102650 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 -- 172.21.15.64:0/3923620426 >> 172.21.15.64:0/3923620426 conn(0x7fa6980fab50 msgr2=0x7fa6980fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 -- 172.21.15.64:0/3923620426 shutdown_connections 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.891+0000 7fa69e27f700 1 -- 172.21.15.64:0/3923620426 wait complete. 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 Processor -- start 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 -- start start 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ff220 0x7fa698101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa6981001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa698102b90 0x7fa6981006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698100e80 con 0x7fa6981021f0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698100ff0 con 0x7fa6980ff220 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa69e27f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6981a0f60 con 0x7fa698102b90 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa6977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa6981001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa6977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa6981001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41410/0 (socket says 172.21.15.64:41410) 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa6977fe700 1 -- 172.21.15.64:0/3918435458 learned_addr learned my addr 172.21.15.64:0/3918435458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.892+0000 7fa697fff700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ff220 0x7fa698101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa69c81c700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa698102b90 0x7fa6981006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6977fe700 1 -- 172.21.15.64:0/3918435458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa698102b90 msgr2=0x7fa6981006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6977fe700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa698102b90 0x7fa6981006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6977fe700 1 -- 172.21.15.64:0/3918435458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ff220 msgr2=0x7fa698101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6977fe700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ff220 0x7fa698101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6977fe700 1 -- 172.21.15.64:0/3918435458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0095f0 con 0x7fa6981021f0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6977fe700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa6981001a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0095c0 tx=0x7fa68c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6957fa700 1 -- 172.21.15.64:0/3918435458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c01d070 con 0x7fa6981021f0 2024-05-08T21:54:32.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6957fa700 1 -- 172.21.15.64:0/3918435458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c022470 con 0x7fa6981021f0 2024-05-08T21:54:32.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6981a1190 con 0x7fa6981021f0 2024-05-08T21:54:32.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa6957fa700 1 -- 172.21.15.64:0/3918435458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c017650 con 0x7fa6981021f0 2024-05-08T21:54:32.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.893+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6981a1550 con 0x7fa6981021f0 2024-05-08T21:54:32.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.895+0000 7fa6957fa700 1 -- 172.21.15.64:0/3918435458 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa68c022890 con 0x7fa6981021f0 2024-05-08T21:54:32.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.896+0000 7fa6957fa700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6800a11b0 0x7fa6800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.896+0000 7fa697fff700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6800a11b0 0x7fa6800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.896+0000 7fa6957fa700 1 -- 172.21.15.64:0/3918435458 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa68c0ce100 con 0x7fa6981021f0 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.896+0000 7fa697fff700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6800a11b0 0x7fa6800a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa698067df0 tx=0x7fa68800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 Processor -- start 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 -- start start 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d24f0 0x556ab41d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d3810 0x556ab41d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab4129100 con 0x556ab41d2e50 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab4126ac0 con 0x556ab41d3810 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab41223b0 con 0x556ab41d24f0 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41422/0 (socket says 172.21.15.64:41422) 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.897+0000 7fa6977fe700 1 -- 172.21.15.64:0/4254977251 learned_addr learned my addr 172.21.15.64:0/4254977251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa69c81c700 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d3810 0x556ab41d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa697fff700 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d24f0 0x556ab41d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa6977fe700 1 -- 172.21.15.64:0/4254977251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d24f0 msgr2=0x556ab41d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa6977fe700 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d24f0 0x556ab41d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa6977fe700 1 -- 172.21.15.64:0/4254977251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d3810 msgr2=0x556ab41d3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa6977fe700 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d3810 0x556ab41d3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa6977fe700 1 -- 172.21.15.64:0/4254977251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ab3fa3360 con 0x556ab41d2e50 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa6977fe700 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41d32d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa68c09b4e0 tx=0x7fa68c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e109f75128fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa67e7fc700 1 -- 172.21.15.64:0/4254977251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c09f170 con 0x556ab41d2e50 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa67e7fc700 1 -- 172.21.15.64:0/4254977251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c09b550 con 0x556ab41d2e50 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.898+0000 7fa67e7fc700 1 -- 172.21.15.64:0/4254977251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c09aa40 con 0x556ab41d2e50 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 -- 172.21.15.64:0/4254977251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 msgr2=0x556ab41d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41d32d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa68c09b4e0 tx=0x7fa68c009310 comp rx=0 tx=0).stop 2024-05-08T21:54:32.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 -- 172.21.15.64:0/4254977251 shutdown_connections 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d3810 0x556ab41d3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d24f0 0x556ab41d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/4254977251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41d32d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 -- 172.21.15.64:0/4254977251 >> 172.21.15.64:0/4254977251 conn(0x556ab41301c0 msgr2=0x556ab41ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 -- 172.21.15.64:0/4254977251 shutdown_connections 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 -- 172.21.15.64:0/4254977251 wait complete. 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 -- start start 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.899+0000 7fa6aa439b80 1 Processor -- start 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab40be510 con 0x556ab41d2e50 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab40be1b0 con 0x556ab41d24f0 2024-05-08T21:54:32.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab3fa5650 con 0x556ab41d3810 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 10 client.0 _ll_register_callbacks cb 0x556ab40c7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41dbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41424/0 (socket says 172.21.15.64:41424) 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 -- 172.21.15.64:0/778963675 learned_addr learned my addr 172.21.15.64:0/778963675 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa697fff700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa69c81c700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 -- 172.21.15.64:0/778963675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 msgr2=0x556ab41e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 -- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 msgr2=0x556ab41dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 -- 172.21.15.64:0/778963675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c097040 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6977fe700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41dbf90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0a3410 tx=0x7fa68c022b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b98a400f65384 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6777fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6777fe700 1 -- 172.21.15.64:0/778963675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0670d0 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6777fe700 1 -- 172.21.15.64:0/778963675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa68c09f380 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.900+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ab407b220 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6777fe700 1 -- 172.21.15.64:0/778963675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa68c09f170 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 10 client.15324 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ab3f4fb10 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6777fe700 1 -- 172.21.15.64:0/778963675 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ab407b220 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6777fe700 1 -- 172.21.15.64:0/778963675 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa68c00fad0 con 0x556ab41d2e50 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:54:32.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 20 client.15324 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab41e6610 0x556ab41e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 4 client.15324 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:32.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ab41a17c0 con 0x556ab41e6610 2024-05-08T21:54:32.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.901+0000 7fa697fff700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab41e6610 0x556ab41e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.902+0000 7fa6777fe700 10 client.15324 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:32.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.902+0000 7fa697fff700 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab41e6610 0x556ab41e4e20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6eec8d291cd93c6 server_cookie=5066afe549b2a1e9 in_seq=0 out_seq=0 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.902+0000 7fa6777fe700 1 -- 172.21.15.64:0/778963675 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ab41a17c0 con 0x556ab41e6610 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.902+0000 7fa6777fe700 10 client.15324 handle_command_reply: tid=0 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 client.15324 shutdown 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 msgr2=0x556ab41dbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41dbf90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0a3410 tx=0x7fa68c022b40 comp rx=0 tx=0).stop 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 shutdown_connections 2024-05-08T21:54:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab41e6610 0x556ab41e4e20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/778963675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d2e50 0x556ab41dbf90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6777fe700 0 client.15324 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 >> 172.21.15.64:0/778963675 conn(0x556ab41301c0 msgr2=0x556ab41cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 shutdown_connections 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 1 -- 172.21.15.64:0/778963675 wait complete. 2024-05-08T21:54:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.903+0000 7fa6aa439b80 20 client.15324 trim_cache size 0 max 16384 2024-05-08T21:54:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 Processor -- start 2024-05-08T21:54:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 -- start start 2024-05-08T21:54:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 0x556ab41d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d2e50 0x556ab41e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41db800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab41a17c0 con 0x556ab41d24f0 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab407b220 con 0x556ab41d2e50 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab411b150 con 0x556ab41d3810 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa697fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 0x556ab41d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa697fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 0x556ab41d9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41436/0 (socket says 172.21.15.64:41436) 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.921+0000 7fa697fff700 1 -- 172.21.15.64:0/2823819905 learned_addr learned my addr 172.21.15.64:0/2823819905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa697fff700 1 -- 172.21.15.64:0/2823819905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 msgr2=0x556ab41db800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa6977fe700 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d2e50 0x556ab41e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa697fff700 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41db800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa697fff700 1 -- 172.21.15.64:0/2823819905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d2e50 msgr2=0x556ab41e2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa697fff700 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d2e50 0x556ab41e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa697fff700 1 -- 172.21.15.64:0/2823819905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ab40e8510 con 0x556ab41d24f0 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa697fff700 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 0x556ab41d9d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa688005cc0 tx=0x7fa688012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb64deaf6952967 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa67effd700 1 -- 172.21.15.64:0/2823819905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688021040 con 0x556ab41d24f0 2024-05-08T21:54:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa67effd700 1 -- 172.21.15.64:0/2823819905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa688014070 con 0x556ab41d24f0 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.922+0000 7fa67effd700 1 -- 172.21.15.64:0/2823819905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688025a90 con 0x556ab41d24f0 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2823819905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 msgr2=0x556ab41d9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 0x556ab41d9d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa688005cc0 tx=0x7fa688012000 comp rx=0 tx=0).stop 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2823819905 shutdown_connections 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d3810 0x556ab41db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d24f0 0x556ab41d9d90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2823819905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d2e50 0x556ab41e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2823819905 >> 172.21.15.64:0/2823819905 conn(0x556ab41301c0 msgr2=0x556ab41da7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2823819905 shutdown_connections 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2823819905 wait complete. 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- start start 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 Processor -- start 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d2e50 0x556ab41dd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 0x556ab41e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab41e2560 con 0x556ab41d3810 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab41e26d0 con 0x556ab41d24f0 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab41e2840 con 0x556ab41d2e50 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.923+0000 7fa6aa439b80 10 client.0 _ll_register_callbacks cb 0x556ab40c7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 0x556ab41e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 0x556ab41e2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41452/0 (socket says 172.21.15.64:41452) 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 -- 172.21.15.64:0/2251825512 learned_addr learned my addr 172.21.15.64:0/2251825512 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa697fff700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6977fe700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d2e50 0x556ab41dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 -- 172.21.15.64:0/2251825512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d2e50 msgr2=0x556ab41dd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d2e50 0x556ab41dd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 -- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 msgr2=0x556ab41dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 -- 172.21.15.64:0/2251825512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68800f980 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa697fff700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa69c81c700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 0x556ab41e2020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa690007af0 tx=0x7fa690007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924c46fed2636452 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6777fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6777fe700 1 -- 172.21.15.64:0/2251825512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690004db0 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6777fe700 1 -- 172.21.15.64:0/2251825512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa690015740 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa68c09b550 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6777fe700 1 -- 172.21.15.64:0/2251825512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa690025030 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6aa439b80 10 client.15330 fetch_fsmap learned FSMap version 27 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.924+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ab3fa7c80 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6777fe700 1 -- 172.21.15.64:0/2251825512 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c09b550 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6777fe700 1 -- 172.21.15.64:0/2251825512 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa69001fb20 con 0x556ab41d3810 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6aa439b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6aa439b80 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:54:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6aa439b80 20 client.15330 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab4242430 0x556ab423ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6aa439b80 4 client.15330 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556ab411ae70 con 0x556ab4242430 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa697fff700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab4242430 0x556ab423ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa697fff700 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab4242430 0x556ab423ed80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1423cbf823b933ff server_cookie=3077888a2d6ccae7 in_seq=0 out_seq=0 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.925+0000 7fa6777fe700 10 client.15330 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6777fe700 1 -- 172.21.15.64:0/2251825512 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556ab411ae70 con 0x556ab4242430 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6777fe700 10 client.15330 handle_command_reply: tid=0 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 client.15330 shutdown 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 msgr2=0x556ab41e2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 0x556ab41e2020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa690007af0 tx=0x7fa690007bd0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 shutdown_connections 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ab41d24f0 0x556ab41dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ab4242430 0x556ab423ed80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ab41d3810 0x556ab41e2020 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 --2- 172.21.15.64:0/2251825512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ab41d2e50 0x556ab41dd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6777fe700 0 client.15330 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 >> 172.21.15.64:0/2251825512 conn(0x556ab41301c0 msgr2=0x556ab41d6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 shutdown_connections 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 1 -- 172.21.15.64:0/2251825512 wait complete. 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa6aa439b80 20 client.15330 trim_cache size 0 max 16384 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "71e641de-0d84-11ef-bc97-c7b262605968", 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24506, 2024-05-08T21:54:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 69.367584387999997, 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 81, 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 75.491413879000007 2024-05-08T21:54:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6800a11b0 msgr2=0x7fa6800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6800a11b0 0x7fa6800a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa698067df0 tx=0x7fa68800a380 comp rx=0 tx=0).stop 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 msgr2=0x7fa6981001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.926+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa6981001a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0095c0 tx=0x7fa68c00f690 comp rx=0 tx=0).stop 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 shutdown_connections 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6800a11b0 0x7fa6800a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6980ff220 0x7fa698101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa698102b90 0x7fa6981006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 --2- 172.21.15.64:0/3918435458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6981021f0 0x7fa6981001a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:32.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 >> 172.21.15.64:0/3918435458 conn(0x7fa6980fab50 msgr2=0x7fa698105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:32.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 shutdown_connections 2024-05-08T21:54:32.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:32.927+0000 7fa69e27f700 1 -- 172.21.15.64:0/3918435458 wait complete. 2024-05-08T21:54:32.939 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T21:54:32.949 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T21:54:32.949 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T21:54:32.949 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T21:54:32.949 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T21:54:33.047 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T21:54:33.047 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T21:54:33.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:33.048 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-08T21:54:33.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:32 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1840722267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:33.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:33.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-05-08T21:54:33.130 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:32 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1840722267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-05-08T21:54:33.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:54:33.138 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-05-08T21:54:33.138 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T21:54:33.138 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-08T21:54:33.138 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-08T21:54:33.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T21:54:33.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:32 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1840722267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:54:33.407 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:54:33.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:33 smithi064 ceph-mon[42500]: pgmap v257: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:34.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:33 smithi160 ceph-mon[46962]: pgmap v257: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:33 smithi186 ceph-mon[46619]: pgmap v257: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 -- 172.21.15.64:0/4212864374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 msgr2=0x7fa0e00fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/4212864374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 0x7fa0e00fff80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0099c0 tx=0x7fa0dc009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 -- 172.21.15.64:0/4212864374 shutdown_connections 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/4212864374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e01063c0 0x7fa0e010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/4212864374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 0x7fa0e00fff80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/4212864374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e0100550 0x7fa0e0105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.406+0000 7fa0e7b48700 1 -- 172.21.15.64:0/4212864374 >> 172.21.15.64:0/4212864374 conn(0x7fa0e0075260 msgr2=0x7fa0e0075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 -- 172.21.15.64:0/4212864374 shutdown_connections 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 -- 172.21.15.64:0/4212864374 wait complete. 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 Processor -- start 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 -- start start 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 0x7fa0e0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e0100550 0x7fa0e0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 0x7fa0e019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e019c200 con 0x7fa0e00ffba0 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e019c370 con 0x7fa0e0100550 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.407+0000 7fa0e7b48700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e019c530 con 0x7fa0e01063c0 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 0x7fa0e019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e50e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e0100550 0x7fa0e0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 0x7fa0e019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53770/0 (socket says 172.21.15.64:53770) 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 -- 172.21.15.64:0/1316371177 learned_addr learned my addr 172.21.15.64:0/1316371177 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e58e4700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 0x7fa0e0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 -- 172.21.15.64:0/1316371177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e0100550 msgr2=0x7fa0e0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e0100550 0x7fa0e0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 -- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 msgr2=0x7fa0e0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 0x7fa0e0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 -- 172.21.15.64:0/1316371177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc0095f0 con 0x7fa0e01063c0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e50e3700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e0100550 0x7fa0e0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.408+0000 7fa0e60e5700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 0x7fa0e019bbf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8007ee0 tx=0x7fa0d800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.409+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8017070 con 0x7fa0e01063c0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.409+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e019c810 con 0x7fa0e01063c0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.409+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d800f040 con 0x7fa0e01063c0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.409+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8005580 con 0x7fa0e01063c0 2024-05-08T21:54:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.409+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e0100c20 con 0x7fa0e01063c0 2024-05-08T21:54:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.409+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e01095a0 con 0x7fa0e01063c0 2024-05-08T21:54:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.411+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa0d800db20 con 0x7fa0e01063c0 2024-05-08T21:54:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.411+0000 7fa0d6ffd700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0cc0a11b0 0x7fa0cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.412+0000 7fa0e58e4700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0cc0a11b0 0x7fa0cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.412+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0d80cc8c0 con 0x7fa0e01063c0 2024-05-08T21:54:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.412+0000 7fa0e58e4700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0cc0a11b0 0x7fa0cc0a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009f50 tx=0x7fa0dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.415+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0d8095ea0 con 0x7fa0e01063c0 2024-05-08T21:54:34.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.663+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa0e0198610 con 0x7fa0e01063c0 2024-05-08T21:54:34.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.663+0000 7fa0d6ffd700 1 -- 172.21.15.64:0/1316371177 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa0d80994c0 con 0x7fa0e01063c0 2024-05-08T21:54:34.664 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T21:54:34.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0cc0a11b0 msgr2=0x7fa0cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:34.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0cc0a11b0 0x7fa0cc0a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009f50 tx=0x7fa0dc005fb0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 msgr2=0x7fa0e019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 0x7fa0e019bbf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8007ee0 tx=0x7fa0d800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 shutdown_connections 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0cc0a11b0 0x7fa0cc0a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e01063c0 0x7fa0e019bbf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e00ffba0 0x7fa0e0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 --2- 172.21.15.64:0/1316371177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e0100550 0x7fa0e0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 >> 172.21.15.64:0/1316371177 conn(0x7fa0e0075260 msgr2=0x7fa0e00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 shutdown_connections 2024-05-08T21:54:34.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:34.666+0000 7fa0e7b48700 1 -- 172.21.15.64:0/1316371177 wait complete. 2024-05-08T21:54:34.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:34 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1316371177' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T21:54:35.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T21:54:35.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T21:54:35.040 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-08T21:54:35.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.040 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-05-08T21:54:35.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:34 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1316371177' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:54:35.059 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:54:35.060 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-05-08T21:54:35.060 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:54:35.060 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-05-08T21:54:35.060 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T21:54:35.060 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-05-08T21:54:35.062 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T21:54:35.062 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41797sec preferred_lft 41797sec 2024-05-08T21:54:35.062 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::2cf5:5cd7:3838:489e/64 scope link noprefixroute 2024-05-08T21:54:35.062 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:54:35.063 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T21:54:35.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.063 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-05-08T21:54:35.063 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-08T21:54:35.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-05-08T21:54:35.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-05-08T21:54:35.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-05-08T21:54:35.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T21:54:35.064 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-08T21:54:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:34 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1316371177' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:54:35.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.237 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T21:54:35.267 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-05-08T21:54:35.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.269 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-05-08T21:54:35.346 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-05-08T21:54:35.346 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T21:54:35.346 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T21:54:35.346 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T21:54:35.346 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T21:54:35.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.347 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-05-08T21:54:35.348 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-08T21:54:35.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T21:54:35.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T21:54:35.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T21:54:35.348 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-08T21:54:35.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.630 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-08T21:54:35.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.647 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-05-08T21:54:35.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.705 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-05-08T21:54:35.705 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-08T21:54:35.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-08T21:54:35.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-08T21:54:35.705 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-08T21:54:35.804 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-08T21:54:35.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:35.804 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-05-08T21:54:35.804 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-08T21:54:35.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-08T21:54:35.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T21:54:35.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-08T21:54:35.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-08T21:54:35.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-08T21:54:35.805 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-08T21:54:35.986 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:35 smithi064 ceph-mon[42500]: pgmap v258: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:36.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.017 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-05-08T21:54:36.017 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-08T21:54:36.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-05-08T21:54:36.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T21:54:36.017 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-08T21:54:36.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:35 smithi160 ceph-mon[46962]: pgmap v258: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:36.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T21:54:36.117 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /tmp/cephfs 2024-05-08T21:54:36.134 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-08T21:54:36.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T21:54:36.135 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /tmp/cephfs 2024-05-08T21:54:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:35 smithi186 ceph-mon[46619]: pgmap v258: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T21:54:36.192 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T21:54:36.192 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T21:54:36.192 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/2e3115a7-b916-4b6b-84de-afba6db85011 2024-05-08T21:54:36.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.193 DEBUG:teuthology.orchestra.run.smithi064:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/2e3115a7-b916-4b6b-84de-afba6db85011 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-05-08T21:54:36.280 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-05-08T21:54:36.280 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-08T21:54:36.280 INFO:teuthology.orchestra.run.smithi064.stdout:invalid new device string format 2024-05-08T21:54:36.373 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-05-08T21:54:36.374 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-08T21:54:36.374 INFO:teuthology.orchestra.run.smithi064.stdout:invalid new device string format 2024-05-08T21:54:36.374 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.160:3300,172.21.15.186:3300" 2024-05-08T21:54:36.374 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.64:3300,172.21.15.160:3300,172.21.15.186:3300:/volumes/qa/sv_1/2e3115a7-b916-4b6b-84de-afba6db85011 2024-05-08T21:54:36.374 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=71e641de-0d84-11ef-bc97-c7b262605968" will pass to kernel 2024-05-08T21:54:36.376 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T21:54:36.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /tmp/cephfs 2024-05-08T21:54:36.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.408 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T21:54:36.501 INFO:teuthology.orchestra.run.smithi064.stdout:client15339 2024-05-08T21:54:36.502 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T21:54:36.502 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T21:54:36.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.520 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T21:54:36.614 INFO:teuthology.orchestra.run.smithi064.stdout:client15339 2024-05-08T21:54:36.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.615 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T21:54:36.667 INFO:teuthology.orchestra.run.smithi064.stdout:client15339 2024-05-08T21:54:36.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:54:36.667 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/71e641de-0d84-11ef-bc97-c7b262605968.client15339/status') 2024-05-08T21:54:36.726 INFO:teuthology.orchestra.run.smithi064.stdout:instance: client.15339 (3)192.168.0.1:0/3861575381 2024-05-08T21:54:36.726 INFO:teuthology.orchestra.run.smithi064.stdout:blocklisted: false 2024-05-08T21:54:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records in 2024-05-08T21:54:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records out 2024-05-08T21:54:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:70 bytes copied, 2.296e-05 s, 3.0 MB/s 2024-05-08T21:54:36.728 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T21:54:36.738 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T21:54:36.738 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:54:36.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-08T21:54:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 -- 172.21.15.64:0/3158919659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ffe00 msgr2=0x7ff84c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 --2- 172.21.15.64:0/3158919659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ffe00 0x7ff84c100220 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0099c0 tx=0x7ff83c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:54:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 -- 172.21.15.64:0/3158919659 shutdown_connections 2024-05-08T21:54:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 --2- 172.21.15.64:0/3158919659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff84c101990 0x7ff84c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 --2- 172.21.15.64:0/3158919659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ffe00 0x7ff84c100220 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 --2- 172.21.15.64:0/3158919659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff84c100ff0 0x7ff84c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 -- 172.21.15.64:0/3158919659 >> 172.21.15.64:0/3158919659 conn(0x7ff84c0fb390 msgr2=0x7ff84c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 -- 172.21.15.64:0/3158919659 shutdown_connections 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.865+0000 7ff8532b8700 1 -- 172.21.15.64:0/3158919659 wait complete. 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 Processor -- start 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 -- start start 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff84c0ffe00 0x7ff84c068da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 0x7ff84c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff84c101990 0x7ff84c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19e940 con 0x7ff84c100ff0 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19eab0 con 0x7ff84c101990 2024-05-08T21:54:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff8532b8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19ec70 con 0x7ff84c0ffe00 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff850853700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 0x7ff84c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff850853700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 0x7ff84c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41488/0 (socket says 172.21.15.64:41488) 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff850853700 1 -- 172.21.15.64:0/1368450327 learned_addr learned my addr 172.21.15.64:0/1368450327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.866+0000 7ff851054700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff84c0ffe00 0x7ff84c068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff851855700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff84c101990 0x7ff84c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff850853700 1 -- 172.21.15.64:0/1368450327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff84c0ffe00 msgr2=0x7ff84c068da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff850853700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff84c0ffe00 0x7ff84c068da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff850853700 1 -- 172.21.15.64:0/1368450327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff84c101990 msgr2=0x7ff84c19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff850853700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff84c101990 0x7ff84c19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff850853700 1 -- 172.21.15.64:0/1368450327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0095f0 con 0x7ff84c100ff0 2024-05-08T21:54:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff850853700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 0x7ff84c19bbb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff84800d950 tx=0x7ff84800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84800f890 con 0x7ff84c100ff0 2024-05-08T21:54:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff848010460 con 0x7ff84c100ff0 2024-05-08T21:54:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84800e610 con 0x7ff84c100ff0 2024-05-08T21:54:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c19ef50 con 0x7ff84c100ff0 2024-05-08T21:54:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.867+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c19f380 con 0x7ff84c100ff0 2024-05-08T21:54:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.868+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c1072d0 con 0x7ff84c100ff0 2024-05-08T21:54:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.869+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff8480108d0 con 0x7ff84c100ff0 2024-05-08T21:54:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.870+0000 7ff8427fc700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff8380a11b0 0x7ff8380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:36.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.870+0000 7ff851054700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff8380a11b0 0x7ff8380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:36.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.870+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8480cc7e0 con 0x7ff84c100ff0 2024-05-08T21:54:36.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.870+0000 7ff851054700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff8380a11b0 0x7ff8380a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff83c003820 tx=0x7ff83c005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:36.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:36.874+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff848095e20 con 0x7ff84c100ff0 2024-05-08T21:54:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.142+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7ff84c067e30 con 0x7ff84c100ff0 2024-05-08T21:54:37.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.739+0000 7ff8427fc700 1 -- 172.21.15.64:0/1368450327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) v1 ==== 117+0+0 (secure 0 0 0) 0x7ff8480160b0 con 0x7ff84c100ff0 2024-05-08T21:54:37.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff8380a11b0 msgr2=0x7ff8380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff8380a11b0 0x7ff8380a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff83c003820 tx=0x7ff83c005920 comp rx=0 tx=0).stop 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 msgr2=0x7ff84c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 0x7ff84c19bbb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff84800d950 tx=0x7ff84800dc60 comp rx=0 tx=0).stop 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 shutdown_connections 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff8380a11b0 0x7ff8380a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff84c101990 0x7ff84c19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff84c0ffe00 0x7ff84c068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 --2- 172.21.15.64:0/1368450327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c100ff0 0x7ff84c19bbb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 >> 172.21.15.64:0/1368450327 conn(0x7ff84c0fb390 msgr2=0x7ff84c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 shutdown_connections 2024-05-08T21:54:37.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T21:54:37.742+0000 7ff8532b8700 1 -- 172.21.15.64:0/1368450327 wait complete. 2024-05-08T21:54:37.757 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T21:54:37.768 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-08T21:54:37.778 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:54:37.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-08T21:54:37.782 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T21:54:37.791 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T21:54:37.791 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T21:54:37.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-08T21:54:37.835 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:37.852 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-08T21:54:37.852 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:3300,172.21.15.160:3300,172.21.15.186:3300:/volumes/qa/sv_1/2e3115a7-b916-4b6b-84de-afba6db85011 340G 0 340G 0% /tmp/cephfs 2024-05-08T21:54:37.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-08T21:54:37.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-08T21:54:37.995 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:37 smithi064 ceph-mon[42500]: pgmap v259: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T21:54:37.995 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:37 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1368450327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T21:54:37.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-08T21:54:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:37 smithi160 ceph-mon[46962]: pgmap v259: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T21:54:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:37 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1368450327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T21:54:38.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:37 smithi186 ceph-mon[46619]: pgmap v259: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T21:54:38.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:37 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1368450327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T21:54:39.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:38 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1368450327' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T21:54:39.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:38 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:39.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:38 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1368450327' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T21:54:39.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:38 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:39.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:38 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1368450327' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T21:54:39.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:38 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:39 smithi160 ceph-mon[46962]: pgmap v260: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T21:54:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:39 smithi186 ceph-mon[46619]: pgmap v260: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T21:54:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:39 smithi064 ceph-mon[42500]: pgmap v260: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T21:54:40.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-08T21:54:40.567 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-08T21:54:40.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-08T21:54:40.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/: 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 .. 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 data 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data: 2024-05-08T21:54:40.733 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:40.734 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:40.734 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 .. 2024-05-08T21:54:40.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-08T21:54:40.824 INFO:teuthology.orchestra.run.smithi064.stdout:[Service] 2024-05-08T21:54:40.824 INFO:teuthology.orchestra.run.smithi064.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-08T21:54:40.824 INFO:teuthology.orchestra.run.smithi064.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-08T21:54:40.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-08T21:54:40.908 INFO:teuthology.orchestra.run.smithi064.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-08T21:54:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:41 smithi160 ceph-mon[46962]: pgmap v261: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-08T21:54:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:41 smithi186 ceph-mon[46619]: pgmap v261: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-08T21:54:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:41 smithi064 ceph-mon[42500]: pgmap v261: 97 pgs: 97 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-08T21:54:44.024 INFO:teuthology.orchestra.run.smithi064.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-08T21:54:44.025 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:43 smithi064 ceph-mon[42500]: pgmap v262: 97 pgs: 97 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T21:54:44.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-08T21:54:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:43 smithi160 ceph-mon[46962]: pgmap v262: 97 pgs: 97 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T21:54:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/: 2024-05-08T21:54:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 .. 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 data 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data: 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:total 30 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 . 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 .. 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 base 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 8 21:54 global 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 log 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_commit_ts 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_dynshmem 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 May 8 21:54 pg_hba.conf 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 8 21:54 pg_ident.conf 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 pg_logical 2024-05-08T21:54:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 pg_multixact 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 pg_notify 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_replslot 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_serial 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_snapshots 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_stat 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_stat_tmp 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 pg_subtrans 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_tblspc 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_twophase 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 pg_wal 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 pg_xact 2024-05-08T21:54:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 8 21:54 postgresql.auto.conf 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 May 8 21:54 postgresql.conf 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base: 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 . 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 1 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 12346 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 12347 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:total 7217 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 . 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 .. 2024-05-08T21:54:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 112 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 113 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 8 21:54 12182 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12182_fsm 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12182_vm 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12184 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12186 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12187_fsm 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187_vm 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12189 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12191 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192 2024-05-08T21:54:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12192_fsm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192_vm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12194 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12196 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12197_fsm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197_vm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12199 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12201 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12202_fsm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202_vm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12204 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12206 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12207_fsm 2024-05-08T21:54:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207_vm 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12209 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12211 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12212 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12214 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12216 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 8 21:54 1247 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1247_fsm 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1247_vm 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 8 21:54 1249 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1249_fsm 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1249_vm 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 8 21:54 1255 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1255_fsm 2024-05-08T21:54:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1255_vm 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 21:54 1259 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1259_fsm 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1259_vm 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417_vm 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418_vm 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 174 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 175 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2187 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224_vm 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328 2024-05-08T21:54:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328_vm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336_vm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2337 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2600 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2600_fsm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2600_vm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2601_fsm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601_vm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2602 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2602_fsm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2602_vm 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2603 2024-05-08T21:54:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2603_fsm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2603_vm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604_vm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2605 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2605_fsm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2605_vm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2606_fsm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606_vm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607_fsm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2607_vm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 8 21:54 2608 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2608_fsm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2608_vm 2024-05-08T21:54:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 8 21:54 2609 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2609_fsm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2609_vm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610_fsm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2610_vm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611_vm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2612_fsm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612_vm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613_vm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2615_fsm 2024-05-08T21:54:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615_vm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616_fsm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2616_vm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 8 21:54 2617 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2617_fsm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2617_vm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 8 21:54 2618 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2618_fsm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2618_vm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 8 21:54 2619 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2619_fsm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2619_vm 2024-05-08T21:54:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620_vm 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2650 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2651 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2652 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2653 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2654 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2655 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2656 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2657 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 8 21:54 2658 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2659 2024-05-08T21:54:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2660 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2661 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2662 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2663 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2664 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2665 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2666 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2667 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2668 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2669 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2670 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 8 21:54 2673 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 May 8 21:54 2674 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 8 21:54 2675 2024-05-08T21:54:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2678 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2679 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2680 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2681 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2682 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2683 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2684 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2685 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2686 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2687 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2688 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2689 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2690 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 8 21:54 2691 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2692 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2693 2024-05-08T21:54:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2696 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2699 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2701 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2702 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2703 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2704 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2753 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2753_fsm 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2753_vm 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2754 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2755 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2756 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2757 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830_vm 2024-05-08T21:54:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2831 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832_vm 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2833 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834_vm 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2835 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836_vm 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2837 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 8 21:54 2838 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2838_fsm 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2838_vm 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2839 2024-05-08T21:54:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840_fsm 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2840_vm 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2841 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995_vm 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2996 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3079_fsm 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079_vm 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3080 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3081 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3085 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118 2024-05-08T21:54:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118_vm 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3119 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3164 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256_vm 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3257 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3258 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350_vm 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3351 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3379 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3380 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381_vm 2024-05-08T21:54:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3394 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3394_fsm 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3394_vm 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3395 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439_vm 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3440 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 3455 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3456 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3456_fsm 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3456_vm 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466_vm 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3467 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3468 2024-05-08T21:54:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501_vm 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3502 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3503 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3534 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3541_fsm 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541_vm 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3542 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3574 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3575 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576_vm 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596_vm 2024-05-08T21:54:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3597 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598_vm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3599 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3600_fsm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600_vm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3601_fsm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601_vm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3602_fsm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602_vm 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3603 2024-05-08T21:54:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3603_fsm 2024-05-08T21:54:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3603_vm 2024-05-08T21:54:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3604 2024-05-08T21:54:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3605 2024-05-08T21:54:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3606 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3607 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3608 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 3609 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3712 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3764_fsm 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764_vm 2024-05-08T21:54:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3766 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3767 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3997 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 5002 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 548 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 549 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102_vm 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104_vm 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106_vm 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6110 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6111 2024-05-08T21:54:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6112 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6113 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6117 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826_vm 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 827 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 828 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/12346: 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:total 7217 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 . 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 .. 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 112 2024-05-08T21:54:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 113 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 8 21:54 12182 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12182_fsm 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12182_vm 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12184 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12186 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12187_fsm 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187_vm 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12189 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12191 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12192_fsm 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192_vm 2024-05-08T21:54:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12194 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12196 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12197_fsm 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197_vm 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12199 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12201 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12202_fsm 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202_vm 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12204 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12206 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12207_fsm 2024-05-08T21:54:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207_vm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12209 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12211 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12212 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12214 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12216 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 8 21:54 1247 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1247_fsm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1247_vm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 8 21:54 1249 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1249_fsm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1249_vm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 8 21:54 1255 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1255_fsm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1255_vm 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 21:54 1259 2024-05-08T21:54:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1259_fsm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1259_vm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417_vm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418_vm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 174 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 175 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2187 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224_vm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328_vm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336_vm 2024-05-08T21:54:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2337 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2600 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2600_fsm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2600_vm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2601_fsm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601_vm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2602 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2602_fsm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2602_vm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2603 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2603_fsm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2603_vm 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604 2024-05-08T21:54:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604_vm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2605 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2605_fsm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2605_vm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2606_fsm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606_vm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607_fsm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2607_vm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 8 21:54 2608 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2608_fsm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2608_vm 2024-05-08T21:54:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 8 21:54 2609 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2609_fsm 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2609_vm 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610_fsm 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2610_vm 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611_vm 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2612_fsm 2024-05-08T21:54:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612_vm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613_vm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2615_fsm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615_vm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616_fsm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2616_vm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 8 21:54 2617 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2617_fsm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2617_vm 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 8 21:54 2618 2024-05-08T21:54:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2618_fsm 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2618_vm 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 8 21:54 2619 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2619_fsm 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2619_vm 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620_vm 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2650 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2651 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2652 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2653 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2654 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2655 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2656 2024-05-08T21:54:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2657 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 8 21:54 2658 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2659 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2660 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2661 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2662 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2663 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2664 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2665 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2666 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2667 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2668 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2669 2024-05-08T21:54:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2670 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 8 21:54 2673 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 May 8 21:54 2674 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 8 21:54 2675 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2678 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2679 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2680 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2681 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2682 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2683 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2684 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2685 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2686 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2687 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2688 2024-05-08T21:54:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2689 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2690 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 8 21:54 2691 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2692 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2693 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2696 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2699 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2701 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2702 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2703 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2704 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2753 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2753_fsm 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2753_vm 2024-05-08T21:54:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2754 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2755 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2756 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2757 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830_vm 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2831 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832_vm 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2833 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834_vm 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2835 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836 2024-05-08T21:54:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836_vm 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2837 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 8 21:54 2838 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2838_fsm 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2838_vm 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2839 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840_fsm 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2840_vm 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2841 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995_vm 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2996 2024-05-08T21:54:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3079_fsm 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079_vm 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3080 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3081 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3085 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118_vm 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3119 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3164 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256_vm 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3257 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3258 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350 2024-05-08T21:54:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350_vm 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3351 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3379 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3380 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381_vm 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3394 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3394_fsm 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3394_vm 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3395 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439_vm 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3440 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 3455 2024-05-08T21:54:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3456 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3456_fsm 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3456_vm 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466_vm 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3467 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3468 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501_vm 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3502 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3503 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3534 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3541_fsm 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541_vm 2024-05-08T21:54:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3542 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3574 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3575 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576_vm 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596_vm 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3597 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598_vm 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3599 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3600_fsm 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600_vm 2024-05-08T21:54:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3601_fsm 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601_vm 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3602_fsm 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602_vm 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3603 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3603_fsm 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3603_vm 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3604 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3605 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3606 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3607 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3608 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 3609 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3712 2024-05-08T21:54:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3764_fsm 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764_vm 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3766 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3767 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3997 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 5002 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 548 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 549 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102_vm 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104_vm 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106_vm 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6110 2024-05-08T21:54:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6111 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6112 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6113 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6117 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826_vm 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 827 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 828 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/12347: 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:total 7217 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 . 2024-05-08T21:54:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 .. 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 112 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 113 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 8 21:54 12182 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12182_fsm 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12182_vm 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12184 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12186 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12187_fsm 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187_vm 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12189 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12191 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192 2024-05-08T21:54:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12192_fsm 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192_vm 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12194 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12196 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12197_fsm 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197_vm 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12199 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12201 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12202_fsm 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202_vm 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12204 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12206 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207 2024-05-08T21:54:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12207_fsm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207_vm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12209 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12211 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12212 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12214 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12216 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 8 21:54 1247 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1247_fsm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1247_vm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 8 21:54 1249 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1249_fsm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1249_vm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 8 21:54 1255 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1255_fsm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1255_vm 2024-05-08T21:54:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 21:54 1259 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1259_fsm 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1259_vm 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417_vm 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418_vm 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 174 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 175 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2187 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224_vm 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328_vm 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336 2024-05-08T21:54:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336_vm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2337 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2600 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2600_fsm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2600_vm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2601_fsm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601_vm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2602 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2602_fsm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2602_vm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2603 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2603_fsm 2024-05-08T21:54:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2603_vm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604_vm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2605 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2605_fsm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2605_vm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2606_fsm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606_vm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607_fsm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2607_vm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 8 21:54 2608 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2608_fsm 2024-05-08T21:54:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2608_vm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 8 21:54 2609 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2609_fsm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2609_vm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610_fsm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2610_vm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611_vm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2612_fsm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612_vm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613_vm 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615 2024-05-08T21:54:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2615_fsm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615_vm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616_fsm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2616_vm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 8 21:54 2617 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2617_fsm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2617_vm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 8 21:54 2618 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2618_fsm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2618_vm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 8 21:54 2619 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2619_fsm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2619_vm 2024-05-08T21:54:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620_vm 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2650 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2651 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2652 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2653 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2654 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2655 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2656 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2657 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 8 21:54 2658 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2659 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2660 2024-05-08T21:54:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2661 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2662 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2663 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2664 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2665 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2666 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2667 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2668 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2669 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2670 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 8 21:54 2673 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 May 8 21:54 2674 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 8 21:54 2675 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2678 2024-05-08T21:54:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2679 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2680 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2681 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2682 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2683 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2684 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2685 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2686 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2687 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2688 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2689 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2690 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 8 21:54 2691 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2692 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2693 2024-05-08T21:54:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2696 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2699 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2701 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2702 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2703 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2704 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2753 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2753_fsm 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2753_vm 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2754 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2755 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2756 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2757 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830 2024-05-08T21:54:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830_vm 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2831 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832_vm 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2833 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834_vm 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2835 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836_vm 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2837 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 8 21:54 2838 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2838_fsm 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2838_vm 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2839 2024-05-08T21:54:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840_fsm 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2840_vm 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2841 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995_vm 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2996 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3079_fsm 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079_vm 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3080 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3081 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3085 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118 2024-05-08T21:54:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118_vm 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3119 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3164 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256_vm 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3257 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3258 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350_vm 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3351 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3379 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3380 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381_vm 2024-05-08T21:54:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3394 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3394_fsm 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3394_vm 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3395 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439_vm 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3440 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 3455 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3456 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3456_fsm 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3456_vm 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466_vm 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3467 2024-05-08T21:54:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3468 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501_vm 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3502 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3503 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3534 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3541_fsm 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541_vm 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3542 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3574 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3575 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576_vm 2024-05-08T21:54:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596_vm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3597 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598_vm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3599 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3600_fsm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600_vm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3601_fsm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601_vm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3602_fsm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602_vm 2024-05-08T21:54:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3603 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3603_fsm 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3603_vm 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3604 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3605 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3606 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3607 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3608 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 3609 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3712 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3764_fsm 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764_vm 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3766 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3767 2024-05-08T21:54:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3997 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 5002 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 548 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 549 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102_vm 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104_vm 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106_vm 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6110 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6111 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6112 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6113 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6117 2024-05-08T21:54:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826_vm 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 827 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 828 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/global: 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:total 553 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 8 21:54 . 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1136 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1136_fsm 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1136_vm 2024-05-08T21:54:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 1137 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1213 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1213_fsm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1213_vm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1214 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1214_fsm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1214_vm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 1232 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 1233 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1260 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1260_fsm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1260_vm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1261 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1261_fsm 2024-05-08T21:54:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1261_vm 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1262 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1262_fsm 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1262_vm 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2396 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2396_fsm 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2396_vm 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2397 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2671 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2672 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2676 2024-05-08T21:54:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2677 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2694 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2695 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2697 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2698 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2846 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2846_vm 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2847 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2964 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2964_vm 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2965 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2966 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2966_vm 2024-05-08T21:54:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2967 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3592 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3592_vm 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3593 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 4060 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 4060_vm 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 4061 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6000 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6000_vm 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6001 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6002 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6100 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6100_vm 2024-05-08T21:54:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6114 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6115 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 pg_control 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/log: 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-08T21:54:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 . 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 mappings 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 8 21:54 replorigin_checkpoint 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 snapshots 2024-05-08T21:54:44.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 .. 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 .. 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 . 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 members 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 offsets 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 .. 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-08T21:54:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 .. 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.151 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.152 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:total 16384 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 . 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 21:54 000000010000000000000001 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 archive_status 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-08T21:54:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 .. 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 8 21:54 .. 2024-05-08T21:54:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T21:54:44.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-08T21:54:44.156 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:43 smithi186 ceph-mon[46619]: pgmap v262: 97 pgs: 97 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T21:54:44.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-08T21:54:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-08T21:54:44.317 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-08T21:54:44.320 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-08T21:54:44.324 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-08T21:54:44.343 INFO:teuthology.orchestra.run.smithi064.stderr:creating tables... 2024-05-08T21:54:44.405 INFO:teuthology.orchestra.run.smithi064.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.88 s) 2024-05-08T21:54:44.590 INFO:teuthology.orchestra.run.smithi064.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.25 s, remaining 3.69 s) 2024-05-08T21:54:44.699 INFO:teuthology.orchestra.run.smithi064.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.35 s, remaining 3.43 s) 2024-05-08T21:54:44.868 INFO:teuthology.orchestra.run.smithi064.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.52 s, remaining 3.66 s) 2024-05-08T21:54:45.046 INFO:teuthology.orchestra.run.smithi064.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.70 s, remaining 3.78 s) 2024-05-08T21:54:45.150 INFO:teuthology.orchestra.run.smithi064.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.81 s, remaining 3.49 s) 2024-05-08T21:54:45.331 INFO:teuthology.orchestra.run.smithi064.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.99 s, remaining 3.52 s) 2024-05-08T21:54:45.503 INFO:teuthology.orchestra.run.smithi064.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.16 s, remaining 3.48 s) 2024-05-08T21:54:45.603 INFO:teuthology.orchestra.run.smithi064.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.26 s, remaining 3.22 s) 2024-05-08T21:54:45.791 INFO:teuthology.orchestra.run.smithi064.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.45 s, remaining 3.18 s) 2024-05-08T21:54:45.968 INFO:teuthology.orchestra.run.smithi064.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.62 s, remaining 3.10 s) 2024-05-08T21:54:46.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:45 smithi160 ceph-mon[46962]: pgmap v263: 97 pgs: 97 active+clean; 20 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2024-05-08T21:54:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.72 s, remaining 2.87 s) 2024-05-08T21:54:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:45 smithi186 ceph-mon[46619]: pgmap v263: 97 pgs: 97 active+clean; 20 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2024-05-08T21:54:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:45 smithi064 ceph-mon[42500]: pgmap v263: 97 pgs: 97 active+clean; 20 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2024-05-08T21:54:46.236 INFO:teuthology.orchestra.run.smithi064.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.89 s, remaining 2.76 s) 2024-05-08T21:54:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:1400000 of 3200000 tuples (43%) done (elapsed 2.08 s, remaining 2.67 s) 2024-05-08T21:54:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:1500000 of 3200000 tuples (46%) done (elapsed 2.18 s, remaining 2.47 s) 2024-05-08T21:54:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.35 s, remaining 2.35 s) 2024-05-08T21:54:46.872 INFO:teuthology.orchestra.run.smithi064.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.53 s, remaining 2.23 s) 2024-05-08T21:54:46.939 INFO:teuthology.orchestra.run.smithi064.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.59 s, remaining 2.02 s) 2024-05-08T21:54:47.145 INFO:teuthology.orchestra.run.smithi064.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.80 s, remaining 1.92 s) 2024-05-08T21:54:47.327 INFO:teuthology.orchestra.run.smithi064.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.98 s, remaining 1.79 s) 2024-05-08T21:54:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2100000 of 3200000 tuples (65%) done (elapsed 3.06 s, remaining 1.60 s) 2024-05-08T21:54:47.611 INFO:teuthology.orchestra.run.smithi064.stderr:2200000 of 3200000 tuples (68%) done (elapsed 3.27 s, remaining 1.48 s) 2024-05-08T21:54:47.684 INFO:teuthology.orchestra.run.smithi064.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.34 s, remaining 1.31 s) 2024-05-08T21:54:47.890 INFO:teuthology.orchestra.run.smithi064.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.55 s, remaining 1.18 s) 2024-05-08T21:54:48.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:47 smithi160 ceph-mon[46962]: pgmap v264: 97 pgs: 97 active+clean; 50 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 5.4 MiB/s wr, 70 op/s 2024-05-08T21:54:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.75 s, remaining 1.05 s) 2024-05-08T21:54:48.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:47 smithi186 ceph-mon[46619]: pgmap v264: 97 pgs: 97 active+clean; 50 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 5.4 MiB/s wr, 70 op/s 2024-05-08T21:54:48.166 INFO:teuthology.orchestra.run.smithi064.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.82 s, remaining 0.88 s) 2024-05-08T21:54:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:47 smithi064 ceph-mon[42500]: pgmap v264: 97 pgs: 97 active+clean; 50 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 5.4 MiB/s wr, 70 op/s 2024-05-08T21:54:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2700000 of 3200000 tuples (84%) done (elapsed 4.00 s, remaining 0.74 s) 2024-05-08T21:54:48.457 INFO:teuthology.orchestra.run.smithi064.stderr:2800000 of 3200000 tuples (87%) done (elapsed 4.11 s, remaining 0.59 s) 2024-05-08T21:54:48.641 INFO:teuthology.orchestra.run.smithi064.stderr:2900000 of 3200000 tuples (90%) done (elapsed 4.30 s, remaining 0.44 s) 2024-05-08T21:54:48.819 INFO:teuthology.orchestra.run.smithi064.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.47 s, remaining 0.30 s) 2024-05-08T21:54:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.58 s, remaining 0.15 s) 2024-05-08T21:54:49.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:48 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:49.090 INFO:teuthology.orchestra.run.smithi064.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.75 s, remaining 0.00 s) 2024-05-08T21:54:49.133 INFO:teuthology.orchestra.run.smithi064.stderr:vacuum... 2024-05-08T21:54:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:48 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:48 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:set primary keys... 2024-05-08T21:54:50.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:49 smithi160 ceph-mon[46962]: pgmap v265: 97 pgs: 97 active+clean; 89 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 11 MiB/s wr, 92 op/s 2024-05-08T21:54:50.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:49 smithi186 ceph-mon[46619]: pgmap v265: 97 pgs: 97 active+clean; 89 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 11 MiB/s wr, 92 op/s 2024-05-08T21:54:50.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:49 smithi064 ceph-mon[42500]: pgmap v265: 97 pgs: 97 active+clean; 89 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 11 MiB/s wr, 92 op/s 2024-05-08T21:54:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 -- 172.21.15.64:0/3605254600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 msgr2=0x7f16d01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/3605254600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d01024f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009990 tx=0x7f16c4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 -- 172.21.15.64:0/3605254600 shutdown_connections 2024-05-08T21:54:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/3605254600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0100f20 0x7f16d0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/3605254600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0102a30 0x7f16d0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/3605254600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d01024f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 -- 172.21.15.64:0/3605254600 >> 172.21.15.64:0/3605254600 conn(0x7f16d00fc490 msgr2=0x7f16d00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 -- 172.21.15.64:0/3605254600 shutdown_connections 2024-05-08T21:54:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.974+0000 7f16d5a9f700 1 -- 172.21.15.64:0/3605254600 wait complete. 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.975+0000 7f16d5a9f700 1 Processor -- start 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.975+0000 7f16d5a9f700 1 -- start start 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16d5a9f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 0x7f16d0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16d5a9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16d5a9f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0102a30 0x7f16d019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16d5a9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d019ea60 con 0x7f16d0102070 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16d5a9f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d019eba0 con 0x7f16d0102a30 2024-05-08T21:54:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16d5a9f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d019ed30 con 0x7f16d0100f20 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 0x7f16d0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 0x7f16d0196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42414/0 (socket says 172.21.15.64:42414) 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16cf7fe700 1 -- 172.21.15.64:0/251073050 learned_addr learned my addr 172.21.15.64:0/251073050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16cffff700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0102a30 0x7f16d019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16ceffd700 1 -- 172.21.15.64:0/251073050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 msgr2=0x7f16d0196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16ceffd700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 0x7f16d0196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16ceffd700 1 -- 172.21.15.64:0/251073050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0102a30 msgr2=0x7f16d019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16ceffd700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0102a30 0x7f16d019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16ceffd700 1 -- 172.21.15.64:0/251073050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40095f0 con 0x7f16d0102070 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 0x7f16d0196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.976+0000 7f16cffff700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0102a30 0x7f16d019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.977+0000 7f16ceffd700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d0197470 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f16c4000c00 tx=0x7f16c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.977+0000 7f16ccff9700 1 -- 172.21.15.64:0/251073050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c401d070 con 0x7f16d0102070 2024-05-08T21:54:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.977+0000 7f16ccff9700 1 -- 172.21.15.64:0/251073050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16c4003f60 con 0x7f16d0102070 2024-05-08T21:54:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.977+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d019efb0 con 0x7f16d0102070 2024-05-08T21:54:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.977+0000 7f16ccff9700 1 -- 172.21.15.64:0/251073050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4022cd0 con 0x7f16d0102070 2024-05-08T21:54:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.977+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d019f3d0 con 0x7f16d0102070 2024-05-08T21:54:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.979+0000 7f16ccff9700 1 -- 172.21.15.64:0/251073050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f16c4003aa0 con 0x7f16d0102070 2024-05-08T21:54:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.980+0000 7f16ccff9700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b80a11a0 0x7f16b80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.980+0000 7f16ccff9700 1 -- 172.21.15.64:0/251073050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16c40cdc40 con 0x7f16d0102070 2024-05-08T21:54:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.980+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b80a11a0 0x7f16b80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.980+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b80a11a0 0x7f16b80a3660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009ea0 tx=0x7f16c0009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.981+0000 7f16e1c59b80 1 Processor -- start 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.981+0000 7f16e1c59b80 1 -- start start 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdcfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cd260f0 con 0x55774cdcf3e0 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cd23ab0 con 0x55774cdd02d0 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cd1f3a0 con 0x55774cdcea80 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdcfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdcfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39680/0 (socket says 172.21.15.64:39680) 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42418/0 (socket says 172.21.15.64:42418) 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 -- 172.21.15.64:0/1774625409 learned_addr learned my addr 172.21.15.64:0/1774625409 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16cffff700 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 -- 172.21.15.64:0/1774625409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 msgr2=0x55774cdceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 -- 172.21.15.64:0/1774625409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 msgr2=0x55774cdd0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 -- 172.21.15.64:0/1774625409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774cba0350 con 0x55774cdcf3e0 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16cffff700 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16ceffd700 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdcfcf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16c409b0c0 tx=0x7f16c4022730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90a4c70c21bbd41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16b5ffb700 1 -- 172.21.15.64:0/1774625409 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c409b830 con 0x55774cdcf3e0 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16b5ffb700 1 -- 172.21.15.64:0/1774625409 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16c409d030 con 0x55774cdcf3e0 2024-05-08T21:54:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.982+0000 7f16b5ffb700 1 -- 172.21.15.64:0/1774625409 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4099580 con 0x55774cdcf3e0 2024-05-08T21:54:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 -- 172.21.15.64:0/1774625409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 msgr2=0x55774cdcfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdcfcf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16c409b0c0 tx=0x7f16c4022730 comp rx=0 tx=0).stop 2024-05-08T21:54:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 -- 172.21.15.64:0/1774625409 shutdown_connections 2024-05-08T21:54:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/1774625409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdcfcf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 -- 172.21.15.64:0/1774625409 >> 172.21.15.64:0/1774625409 conn(0x55774cd2d1b0 msgr2=0x55774cdcb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 -- 172.21.15.64:0/1774625409 shutdown_connections 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.983+0000 7f16e1c59b80 1 -- 172.21.15.64:0/1774625409 wait complete. 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 -- start start 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 Processor -- start 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdd8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cdd8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdd02d0 0x55774cddfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ccbb500 con 0x55774cdd02d0 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ccbb1a0 con 0x55774cdcf3e0 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cba2640 con 0x55774cdcea80 2024-05-08T21:54:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16e1c59b80 10 client.0 _ll_register_callbacks cb 0x55774ccc4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdd8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdd02d0 0x55774cddfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdd8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42430/0 (socket says 172.21.15.64:42430) 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.984+0000 7f16cf7fe700 1 -- 172.21.15.64:0/878977273 learned_addr learned my addr 172.21.15.64:0/878977273 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cffff700 1 -- 172.21.15.64:0/878977273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 msgr2=0x55774cdd8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cffff700 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdd8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cffff700 1 -- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 msgr2=0x55774cdd8a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cffff700 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cdd8a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cffff700 1 -- 172.21.15.64:0/878977273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d019f3d0 con 0x55774cdd02d0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdd8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16cffff700 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdd02d0 0x55774cddfac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16c8007fc0 tx=0x7f16c8012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb90305ce08bbf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16aeffd700 1 -- 172.21.15.64:0/878977273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8003a00 con 0x55774cdd02d0 2024-05-08T21:54:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16aeffd700 1 -- 172.21.15.64:0/878977273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16c8003b60 con 0x55774cdd02d0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55774cc78210 con 0x55774cdd02d0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16aeffd700 1 -- 172.21.15.64:0/878977273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16c8003d20 con 0x55774cdd02d0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16e1c59b80 10 client.15357 fetch_fsmap learned FSMap version 29 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.985+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55774cb4cb00 con 0x55774cdd02d0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16aeffd700 1 -- 172.21.15.64:0/878977273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16c8027020 con 0x55774cdd02d0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16aeffd700 1 -- 172.21.15.64:0/878977273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f16c801f060 con 0x55774cdd02d0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16e1c59b80 10 client.15357 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16e1c59b80 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16e1c59b80 20 client.15357 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774cde4450 0x55774cde6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16e1c59b80 4 client.15357 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55774cd9e7b0 con 0x55774cde4450 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.986+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774cde4450 0x55774cde6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16aeffd700 10 client.15357 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774cde4450 0x55774cde6870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06d55d6528a039b server_cookie=81a4664619ddb2c2 in_seq=0 out_seq=0 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16aeffd700 1 -- 172.21.15.64:0/878977273 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55774cd9e7b0 con 0x55774cde4450 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16aeffd700 10 client.15357 handle_command_reply: tid=0 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 client.15357 shutdown 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdd02d0 msgr2=0x55774cddfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdd02d0 0x55774cddfac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16c8007fc0 tx=0x7f16c8012520 comp rx=0 tx=0).stop 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 shutdown_connections 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774cde4450 0x55774cde6870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdd02d0 0x55774cddfac0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16aeffd700 0 client.15357 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cdd8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/878977273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cdd8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.987+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 >> 172.21.15.64:0/878977273 conn(0x55774cd2d1b0 msgr2=0x55774cdc9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.988+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 shutdown_connections 2024-05-08T21:54:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.988+0000 7f16e1c59b80 1 -- 172.21.15.64:0/878977273 wait complete. 2024-05-08T21:54:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:50.988+0000 7f16e1c59b80 20 client.15357 trim_cache size 0 max 16384 2024-05-08T21:54:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 Processor -- start 2024-05-08T21:54:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 -- start start 2024-05-08T21:54:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cde0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdd7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cdeba80 con 0x55774cdcf3e0 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cdd6490 con 0x55774cdd02d0 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cdd3d80 con 0x55774cdcea80 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdd7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16cffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdd7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39702/0 (socket says 172.21.15.64:39702) 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16ceffd700 1 -- 172.21.15.64:0/2744568768 learned_addr learned my addr 172.21.15.64:0/2744568768 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.003+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cde0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16ceffd700 1 -- 172.21.15.64:0/2744568768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 msgr2=0x55774cde0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16ceffd700 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cde0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16ceffd700 1 -- 172.21.15.64:0/2744568768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 msgr2=0x55774cdd06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16ceffd700 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16ceffd700 1 -- 172.21.15.64:0/2744568768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774cce5500 con 0x55774cdcf3e0 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16ceffd700 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdd7a50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16c40a2ff0 tx=0x7f16c40a3520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f9134af0e25d93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16b67fc700 1 -- 172.21.15.64:0/2744568768 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c409b830 con 0x55774cdcf3e0 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16b67fc700 1 -- 172.21.15.64:0/2744568768 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16c409b130 con 0x55774cdcf3e0 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.004+0000 7f16b67fc700 1 -- 172.21.15.64:0/2744568768 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c40a10c0 con 0x55774cdcf3e0 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2744568768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 msgr2=0x55774cdd7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdd7a50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16c40a2ff0 tx=0x7f16c40a3520 comp rx=0 tx=0).stop 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2744568768 shutdown_connections 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdd02d0 0x55774cdd06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdcea80 0x55774cde0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2744568768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcf3e0 0x55774cdd7a50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2744568768 >> 172.21.15.64:0/2744568768 conn(0x55774cd2d1b0 msgr2=0x55774cdd6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2744568768 shutdown_connections 2024-05-08T21:54:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2744568768 wait complete. 2024-05-08T21:54:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 -- start start 2024-05-08T21:54:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 Processor -- start 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 0x55774cddcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.005+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cddd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16e1c59b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdd02d0 0x55774cdde4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cdde9f0 con 0x55774cdcea80 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cddeb60 con 0x55774cdcf3e0 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16e1c59b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774cdded20 con 0x55774cdd02d0 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16e1c59b80 10 client.0 _ll_register_callbacks cb 0x55774ccc4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 0x55774cddcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 0x55774cddcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39704/0 (socket says 172.21.15.64:39704) 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 -- 172.21.15.64:0/2070709896 learned_addr learned my addr 172.21.15.64:0/2070709896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:54:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16ceffd700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cddd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cffff700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdd02d0 0x55774cdde4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 -- 172.21.15.64:0/2070709896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdd02d0 msgr2=0x55774cdde4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdd02d0 0x55774cdde4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 -- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 msgr2=0x55774cddd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cddd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 -- 172.21.15.64:0/2070709896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774cba0350 con 0x55774cdcea80 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16cf7fe700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 0x55774cddcf60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55774cde7120 tx=0x7f16c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec3386b6a4e04d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16aeffd700 1 -- 172.21.15.64:0/2070709896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0027050 con 0x55774cdcea80 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16aeffd700 1 -- 172.21.15.64:0/2070709896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16c0022640 con 0x55774cdcea80 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.006+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16c409d030 con 0x55774cdcea80 2024-05-08T21:54:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16aeffd700 1 -- 172.21.15.64:0/2070709896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16c0022800 con 0x55774cdcea80 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 10 client.15363 fetch_fsmap learned FSMap version 29 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55774cba4c70 con 0x55774cdcea80 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16aeffd700 1 -- 172.21.15.64:0/2070709896 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16c003d020 con 0x55774cdcea80 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16aeffd700 1 -- 172.21.15.64:0/2070709896 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f16c00399b0 con 0x55774cdcea80 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 20 client.15363 populate_metadata read hostname 'smithi064' 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774ce3b070 0x55774ce3d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 4 client.15363 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55774cd17e60 con 0x55774ce3b070 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.007+0000 7f16ceffd700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774ce3b070 0x55774ce3d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.008+0000 7f16aeffd700 10 client.15363 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.008+0000 7f16ceffd700 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774ce3b070 0x55774ce3d490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1afefe2e4b0729 server_cookie=f9bdaceec0561c65 in_seq=0 out_seq=0 2024-05-08T21:54:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.157+0000 7f16aeffd700 1 -- 172.21.15.64:0/2070709896 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55774cd17e60 con 0x55774ce3b070 2024-05-08T21:54:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.157+0000 7f16aeffd700 10 client.15363 handle_command_reply: tid=0 2024-05-08T21:54:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 client.15363 shutdown 2024-05-08T21:54:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 msgr2=0x55774cddcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 0x55774cddcf60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55774cde7120 tx=0x7f16c0004750 comp rx=0 tx=0).stop 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 shutdown_connections 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55774ce3b070 0x55774ce3d490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774cdd02d0 0x55774cdde4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55774cdcea80 0x55774cddcf60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 --2- 172.21.15.64:0/2070709896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55774cdcf3e0 0x55774cddd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.160+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 >> 172.21.15.64:0/2070709896 conn(0x55774cd2d1b0 msgr2=0x55774cdcb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.162+0000 7f16aeffd700 0 client.15363 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:54:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.162+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 shutdown_connections 2024-05-08T21:54:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.162+0000 7f16e1c59b80 1 -- 172.21.15.64:0/2070709896 wait complete. 2024-05-08T21:54:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.162+0000 7f16e1c59b80 20 client.15363 trim_cache size 0 max 16384 2024-05-08T21:54:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:54:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:54:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:54:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:54:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b80a11a0 msgr2=0x7f16b80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b80a11a0 0x7f16b80a3660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009ea0 tx=0x7f16c0009290 comp rx=0 tx=0).stop 2024-05-08T21:54:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 msgr2=0x7f16d0197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:54:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d0197470 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f16c4000c00 tx=0x7f16c4004730 comp rx=0 tx=0).stop 2024-05-08T21:54:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 shutdown_connections 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16b80a11a0 0x7f16b80a3660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0102a30 0x7f16d019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16d0100f20 0x7f16d0196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 --2- 172.21.15.64:0/251073050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d0102070 0x7f16d0197470 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 >> 172.21.15.64:0/251073050 conn(0x7f16d00fc490 msgr2=0x7f16d01054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 shutdown_connections 2024-05-08T21:54:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:54:51.163+0000 7f16d5a9f700 1 -- 172.21.15.64:0/251073050 wait complete. 2024-05-08T21:54:51.409 INFO:teuthology.orchestra.run.smithi064.stderr:done. 2024-05-08T21:54:51.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-08T21:54:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:starting vacuum...end. 2024-05-08T21:54:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:51 smithi186 ceph-mon[46619]: pgmap v266: 97 pgs: 97 active+clean; 263 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 39 MiB/s wr, 119 op/s 2024-05-08T21:54:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:51 smithi186 ceph-mon[46619]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T21:54:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:51 smithi064 ceph-mon[42500]: pgmap v266: 97 pgs: 97 active+clean; 263 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 39 MiB/s wr, 119 op/s 2024-05-08T21:54:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:51 smithi064 ceph-mon[42500]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T21:54:52.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:51 smithi160 ceph-mon[46962]: pgmap v266: 97 pgs: 97 active+clean; 263 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 39 MiB/s wr, 119 op/s 2024-05-08T21:54:52.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:51 smithi160 ceph-mon[46962]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T21:54:54.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:53 smithi186 ceph-mon[46619]: pgmap v267: 97 pgs: 97 active+clean; 363 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 55 MiB/s wr, 161 op/s 2024-05-08T21:54:54.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:53 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:54.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:53 smithi064 ceph-mon[42500]: pgmap v267: 97 pgs: 97 active+clean; 363 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 55 MiB/s wr, 161 op/s 2024-05-08T21:54:54.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:53 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:54.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:53 smithi160 ceph-mon[46962]: pgmap v267: 97 pgs: 97 active+clean; 363 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 55 MiB/s wr, 161 op/s 2024-05-08T21:54:54.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:53 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:54:56.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:55 smithi186 ceph-mon[46619]: pgmap v268: 97 pgs: 97 active+clean; 689 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 77 MiB/s wr, 334 op/s 2024-05-08T21:54:56.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:55 smithi064 ceph-mon[42500]: pgmap v268: 97 pgs: 97 active+clean; 689 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 77 MiB/s wr, 334 op/s 2024-05-08T21:54:56.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:55 smithi160 ceph-mon[46962]: pgmap v268: 97 pgs: 97 active+clean; 689 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 77 MiB/s wr, 334 op/s 2024-05-08T21:54:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 5.0 s, 3722.4 tps, lat 26.424 ms stddev 34.054 2024-05-08T21:54:58.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:57 smithi186 ceph-mon[46619]: pgmap v269: 97 pgs: 97 active+clean; 841 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 86 MiB/s wr, 425 op/s 2024-05-08T21:54:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:57 smithi064 ceph-mon[42500]: pgmap v269: 97 pgs: 97 active+clean; 841 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 86 MiB/s wr, 425 op/s 2024-05-08T21:54:58.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:57 smithi160 ceph-mon[46962]: pgmap v269: 97 pgs: 97 active+clean; 841 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 86 MiB/s wr, 425 op/s 2024-05-08T21:55:00.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:54:59 smithi186 ceph-mon[46619]: pgmap v270: 97 pgs: 97 active+clean; 1010 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 MiB/s wr, 558 op/s 2024-05-08T21:55:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:54:59 smithi064 ceph-mon[42500]: pgmap v270: 97 pgs: 97 active+clean; 1010 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 MiB/s wr, 558 op/s 2024-05-08T21:55:00.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:54:59 smithi160 ceph-mon[46962]: pgmap v270: 97 pgs: 97 active+clean; 1010 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 MiB/s wr, 558 op/s 2024-05-08T21:55:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 10.0 s, 3906.9 tps, lat 25.376 ms stddev 47.252 2024-05-08T21:55:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:01 smithi186 ceph-mon[46619]: pgmap v271: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 MiB/s wr, 859 op/s 2024-05-08T21:55:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:01 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:02.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:01 smithi064 ceph-mon[42500]: pgmap v271: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 MiB/s wr, 859 op/s 2024-05-08T21:55:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:01 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:01 smithi160 ceph-mon[46962]: pgmap v271: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 MiB/s wr, 859 op/s 2024-05-08T21:55:02.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:01 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:04.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:03 smithi186 ceph-mon[46619]: pgmap v272: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 114 MiB/s wr, 1.09k op/s 2024-05-08T21:55:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:03 smithi064 ceph-mon[42500]: pgmap v272: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 114 MiB/s wr, 1.09k op/s 2024-05-08T21:55:04.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:03 smithi160 ceph-mon[46962]: pgmap v272: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 114 MiB/s wr, 1.09k op/s 2024-05-08T21:55:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.327+0000 7f7762c38700 1 -- 172.21.15.64:0/1075071241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c103140 msgr2=0x7f775c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.327+0000 7f7762c38700 1 --2- 172.21.15.64:0/1075071241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c103140 0x7f775c103560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7750009990 tx=0x7f7750009ca0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 -- 172.21.15.64:0/1075071241 shutdown_connections 2024-05-08T21:55:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 --2- 172.21.15.64:0/1075071241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f775c104cd0 0x7f775c10b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 --2- 172.21.15.64:0/1075071241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c103140 0x7f775c103560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 --2- 172.21.15.64:0/1075071241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c104330 0x7f775c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 -- 172.21.15.64:0/1075071241 >> 172.21.15.64:0/1075071241 conn(0x7f775c0fe6b0 msgr2=0x7f775c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 -- 172.21.15.64:0/1075071241 shutdown_connections 2024-05-08T21:55:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.328+0000 7f7762c38700 1 -- 172.21.15.64:0/1075071241 wait complete. 2024-05-08T21:55:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 Processor -- start 2024-05-08T21:55:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 -- start start 2024-05-08T21:55:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 0x7f775c199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104330 0x7f775c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f775c104cd0 0x7f775c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1a0ba0 con 0x7f775c103140 2024-05-08T21:55:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1a0ce0 con 0x7f775c104cd0 2024-05-08T21:55:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f7762c38700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1a0e70 con 0x7f775c104330 2024-05-08T21:55:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f77609d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 0x7f775c199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f77611d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f775c104cd0 0x7f775c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f77609d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 0x7f775c199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49792/0 (socket says 172.21.15.64:49792) 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.329+0000 7f77609d4700 1 -- 172.21.15.64:0/1321883620 learned_addr learned my addr 172.21.15.64:0/1321883620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f775bfff700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104330 0x7f775c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f77609d4700 1 -- 172.21.15.64:0/1321883620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104330 msgr2=0x7f775c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f77609d4700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104330 0x7f775c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f77609d4700 1 -- 172.21.15.64:0/1321883620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f775c104cd0 msgr2=0x7f775c1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f77609d4700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f775c104cd0 0x7f775c1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f77609d4700 1 -- 172.21.15.64:0/1321883620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500095f0 con 0x7f775c103140 2024-05-08T21:55:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.330+0000 7f77609d4700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 0x7f775c199070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77500098e0 tx=0x7f7750005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.331+0000 7f7759ffb700 1 -- 172.21.15.64:0/1321883620 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775001d070 con 0x7f775c103140 2024-05-08T21:55:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.331+0000 7f7759ffb700 1 -- 172.21.15.64:0/1321883620 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7750022470 con 0x7f775c103140 2024-05-08T21:55:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.331+0000 7f7759ffb700 1 -- 172.21.15.64:0/1321883620 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750017650 con 0x7f775c103140 2024-05-08T21:55:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.331+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c1a10f0 con 0x7f775c103140 2024-05-08T21:55:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.331+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c0755c0 con 0x7f775c103140 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.337+0000 7f776edf2b80 1 Processor -- start 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 -- start start 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721c9dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9e3e0 0x55e721c9ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 0x55e721c9f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f7759ffb700 1 -- 172.21.15.64:0/1321883620 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7750022850 con 0x7f775c103140 2024-05-08T21:55:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721bf50f0 con 0x55e721c9e3e0 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721bf2ab0 con 0x55e721c9da80 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f776edf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721bee3a0 con 0x55e721c9f2d0 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f775bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 0x55e721c9f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f775bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 0x55e721c9f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:52794/0 (socket says 172.21.15.64:52794) 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.345+0000 7f775bfff700 1 -- 172.21.15.64:0/407645768 learned_addr learned my addr 172.21.15.64:0/407645768 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f775bfff700 1 -- 172.21.15.64:0/407645768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 msgr2=0x55e721c9dea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f775bfff700 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721c9dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f775bfff700 1 -- 172.21.15.64:0/407645768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9e3e0 msgr2=0x55e721c9ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f775bfff700 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9e3e0 0x55e721c9ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f775bfff700 1 -- 172.21.15.64:0/407645768 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e721a6f350 con 0x55e721c9f2d0 2024-05-08T21:55:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f775bfff700 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 0x55e721c9f750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7748009d80 tx=0x7f7748011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b2cb4721b5bac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f7759ffb700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f774c0a9930 0x7f774c0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f7746ffd700 1 -- 172.21.15.64:0/407645768 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748004230 con 0x55e721c9f2d0 2024-05-08T21:55:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.346+0000 7f7759ffb700 1 -- 172.21.15.64:0/1321883620 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77500cdd40 con 0x7f775c103140 2024-05-08T21:55:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.348+0000 7f7746ffd700 1 -- 172.21.15.64:0/407645768 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7748004390 con 0x55e721c9f2d0 2024-05-08T21:55:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.348+0000 7f7746ffd700 1 -- 172.21.15.64:0/407645768 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748019820 con 0x55e721c9f2d0 2024-05-08T21:55:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f77611d5700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f774c0a9930 0x7f774c0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 -- 172.21.15.64:0/407645768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 msgr2=0x55e721c9f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 0x55e721c9f750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7748009d80 tx=0x7f7748011720 comp rx=0 tx=0).stop 2024-05-08T21:55:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f77611d5700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f774c0a9930 0x7f774c0abdf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f775400f520 tx=0x7f775400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 -- 172.21.15.64:0/407645768 shutdown_connections 2024-05-08T21:55:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721c9dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9f2d0 0x55e721c9f750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 --2- 172.21.15.64:0/407645768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9e3e0 0x55e721c9ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 -- 172.21.15.64:0/407645768 >> 172.21.15.64:0/407645768 conn(0x55e721bfc1b0 msgr2=0x55e721c9a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 -- 172.21.15.64:0/407645768 shutdown_connections 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.349+0000 7f776edf2b80 1 -- 172.21.15.64:0/407645768 wait complete. 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.350+0000 7f776edf2b80 1 -- start start 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.350+0000 7f776edf2b80 1 Processor -- start 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.350+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.350+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721ca7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721caede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f776edf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721b8a500 con 0x55e721c9f2d0 2024-05-08T21:55:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f776edf2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721b8a1a0 con 0x55e721c9da80 2024-05-08T21:55:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f776edf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721a71640 con 0x55e721c9e3e0 2024-05-08T21:55:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f776edf2b80 10 client.0 _ll_register_callbacks cb 0x55e721b93ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f77609d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f77609d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:52514/0 (socket says 172.21.15.64:52514) 2024-05-08T21:55:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f77609d4700 1 -- 172.21.15.64:0/151685074 learned_addr learned my addr 172.21.15.64:0/151685074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721ca7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 -- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 msgr2=0x55e721ca7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 -- 172.21.15.64:0/151685074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 msgr2=0x55e721caede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721caede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 -- 172.21.15.64:0/151685074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77480099e0 con 0x55e721c9e3e0 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f77609d4700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f775bfff700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721ca7da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7748011b20 tx=0x7f7748003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1722c22b79a3345 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f773bfff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.351+0000 7f773bfff700 1 -- 172.21.15.64:0/151685074 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748005450 con 0x55e721c9e3e0 2024-05-08T21:55:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.352+0000 7f773bfff700 1 -- 172.21.15.64:0/151685074 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7748004500 con 0x55e721c9e3e0 2024-05-08T21:55:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.354+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e721b47210 con 0x55e721c9e3e0 2024-05-08T21:55:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.355+0000 7f773bfff700 1 -- 172.21.15.64:0/151685074 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7748003c00 con 0x55e721c9e3e0 2024-05-08T21:55:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 10 client.25021 fetch_fsmap learned FSMap version 31 2024-05-08T21:55:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e721a1bb00 con 0x55e721c9e3e0 2024-05-08T21:55:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f773bfff700 1 -- 172.21.15.64:0/151685074 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e721b47210 con 0x55e721c9e3e0 2024-05-08T21:55:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f773bfff700 1 -- 172.21.15.64:0/151685074 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7748021040 con 0x55e721c9e3e0 2024-05-08T21:55:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 10 client.25021 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:55:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 10 client.25021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 20 client.25021 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721cb37c0 0x55e721cb5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 4 client.25021 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.356+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e721c6d7b0 con 0x55e721cb37c0 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.357+0000 7f77609d4700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721cb37c0 0x55e721cb5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.357+0000 7f773bfff700 10 client.25021 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.357+0000 7f77609d4700 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721cb37c0 0x55e721cb5be0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1215c992642f15 server_cookie=71780161b5670751 in_seq=0 out_seq=0 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f773bfff700 1 -- 172.21.15.64:0/151685074 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e721c6d7b0 con 0x55e721cb37c0 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f773bfff700 10 client.25021 handle_command_reply: tid=0 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 client.25021 shutdown 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 msgr2=0x55e721ca7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721ca7da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7748011b20 tx=0x7f7748003960 comp rx=0 tx=0).stop 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 shutdown_connections 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721cb37c0 0x55e721cb5be0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721caede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 --2- 172.21.15.64:0/151685074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721ca7da0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f773bfff700 0 client.25021 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 >> 172.21.15.64:0/151685074 conn(0x55e721bfc1b0 msgr2=0x55e721c98860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 shutdown_connections 2024-05-08T21:55:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.358+0000 7f776edf2b80 1 -- 172.21.15.64:0/151685074 wait complete. 2024-05-08T21:55:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.359+0000 7f776edf2b80 20 client.25021 trim_cache size 0 max 16384 2024-05-08T21:55:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 Processor -- start 2024-05-08T21:55:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 -- start start 2024-05-08T21:55:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721c9f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721c9e7c0 con 0x55e721c9f2d0 2024-05-08T21:55:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721ca8a50 con 0x55e721c9da80 2024-05-08T21:55:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.383+0000 7f776edf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721cbad90 con 0x55e721c9e3e0 2024-05-08T21:55:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:52814/0 (socket says 172.21.15.64:52814) 2024-05-08T21:55:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 -- 172.21.15.64:0/1504820858 learned_addr learned my addr 172.21.15.64:0/1504820858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f77611d5700 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721c9f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 -- 172.21.15.64:0/1504820858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 msgr2=0x55e721ca6de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:55:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f77609d4700 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 -- 172.21.15.64:0/1504820858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 msgr2=0x55e721c9f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721c9f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 -- 172.21.15.64:0/1504820858 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e721bb4500 con 0x55e721c9e3e0 2024-05-08T21:55:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f77611d5700 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721c9f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:55:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f775bfff700 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caeda0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7748011760 tx=0x7f7748003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a685f6732fd8ce3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.384+0000 7f77477fe700 1 -- 172.21.15.64:0/1504820858 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748019810 con 0x55e721c9e3e0 2024-05-08T21:55:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.385+0000 7f77477fe700 1 -- 172.21.15.64:0/1504820858 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7748003a80 con 0x55e721c9e3e0 2024-05-08T21:55:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.385+0000 7f77477fe700 1 -- 172.21.15.64:0/1504820858 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748035670 con 0x55e721c9e3e0 2024-05-08T21:55:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 -- 172.21.15.64:0/1504820858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 msgr2=0x55e721caeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caeda0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7748011760 tx=0x7f7748003ee0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 -- 172.21.15.64:0/1504820858 shutdown_connections 2024-05-08T21:55:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9da80 0x55e721ca6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9f2d0 0x55e721c9f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 --2- 172.21.15.64:0/1504820858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caeda0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 -- 172.21.15.64:0/1504820858 >> 172.21.15.64:0/1504820858 conn(0x55e721bfc1b0 msgr2=0x55e721ca5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 -- 172.21.15.64:0/1504820858 shutdown_connections 2024-05-08T21:55:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 -- 172.21.15.64:0/1504820858 wait complete. 2024-05-08T21:55:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 -- start start 2024-05-08T21:55:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.386+0000 7f776edf2b80 1 Processor -- start 2024-05-08T21:55:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9da80 0x55e721caa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 0x55e721cad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721cadbc0 con 0x55e721c9da80 2024-05-08T21:55:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721cadd30 con 0x55e721c9f2d0 2024-05-08T21:55:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e721cadef0 con 0x55e721c9e3e0 2024-05-08T21:55:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f776edf2b80 10 client.0 _ll_register_callbacks cb 0x55e721b93ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.387+0000 7f77611d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 0x55e721cad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f775bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 0x55e721cad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:52540/0 (socket says 172.21.15.64:52540) 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 -- 172.21.15.64:0/3127660854 learned_addr learned my addr 172.21.15.64:0/3127660854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77609d4700 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9da80 0x55e721caa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 -- 172.21.15.64:0/3127660854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 msgr2=0x55e721caa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 -- 172.21.15.64:0/3127660854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9da80 msgr2=0x55e721caa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9da80 0x55e721caa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.388+0000 7f77611d5700 1 -- 172.21.15.64:0/3127660854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e721a1bb00 con 0x55e721c9f2d0 2024-05-08T21:55:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f77611d5700 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 0x55e721cad680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7754008a90 tx=0x7f7754009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c2d7c70ac0f049 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f773bfff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:55:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f773bfff700 1 -- 172.21.15.64:0/3127660854 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400e040 con 0x55e721c9f2d0 2024-05-08T21:55:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7748004390 con 0x55e721c9f2d0 2024-05-08T21:55:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f773bfff700 1 -- 172.21.15.64:0/3127660854 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7754017a50 con 0x55e721c9f2d0 2024-05-08T21:55:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f773bfff700 1 -- 172.21.15.64:0/3127660854 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7754009400 con 0x55e721c9f2d0 2024-05-08T21:55:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f776edf2b80 10 client.25076 fetch_fsmap learned FSMap version 31 2024-05-08T21:55:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.389+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e721a73c70 con 0x55e721c9f2d0 2024-05-08T21:55:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f773bfff700 1 -- 172.21.15.64:0/3127660854 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7748004390 con 0x55e721c9f2d0 2024-05-08T21:55:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f773bfff700 1 -- 172.21.15.64:0/3127660854 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7754022c60 con 0x55e721c9f2d0 2024-05-08T21:55:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f776edf2b80 10 client.25076 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:55:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f776edf2b80 10 client.25076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f776edf2b80 20 client.25076 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f776edf2b80 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721d1b240 0x55e721d1d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f776edf2b80 4 client.25076 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:55:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e721be6e60 con 0x55e721d1b240 2024-05-08T21:55:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.390+0000 7f77609d4700 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721d1b240 0x55e721d1d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.391+0000 7f773bfff700 10 client.25076 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.391+0000 7f77609d4700 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721d1b240 0x55e721d1d660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f30a26aef2be8705 server_cookie=7887041f6e25137b in_seq=0 out_seq=0 2024-05-08T21:55:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.407+0000 7f773bfff700 1 -- 172.21.15.64:0/3127660854 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e721be6e60 con 0x55e721d1b240 2024-05-08T21:55:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.407+0000 7f773bfff700 10 client.25076 handle_command_reply: tid=0 2024-05-08T21:55:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.407+0000 7f776edf2b80 1 client.25076 shutdown 2024-05-08T21:55:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.407+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 msgr2=0x55e721cad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.407+0000 7f776edf2b80 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 0x55e721cad680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7754008a90 tx=0x7f7754009920 comp rx=0 tx=0).stop 2024-05-08T21:55:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 shutdown_connections 2024-05-08T21:55:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f776edf2b80 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e721c9f2d0 0x55e721cad680 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f776edf2b80 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e721d1b240 0x55e721d1d660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f776edf2b80 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e721c9da80 0x55e721caa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f776edf2b80 1 --2- 172.21.15.64:0/3127660854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e721c9e3e0 0x55e721caa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 >> 172.21.15.64:0/3127660854 conn(0x55e721bfc1b0 msgr2=0x55e721c9a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.408+0000 7f773bfff700 0 client.25076 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.409+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 shutdown_connections 2024-05-08T21:55:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:55:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:55:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:55:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.409+0000 7f776edf2b80 1 -- 172.21.15.64:0/3127660854 wait complete. 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.410+0000 7f776edf2b80 20 client.25076 trim_cache size 0 max 16384 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f774c0a9930 msgr2=0x7f774c0abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f774c0a9930 0x7f774c0abdf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f775400f520 tx=0x7f775400b040 comp rx=0 tx=0).stop 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 msgr2=0x7f775c199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 0x7f775c199070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77500098e0 tx=0x7f7750005740 comp rx=0 tx=0).stop 2024-05-08T21:55:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 shutdown_connections 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f774c0a9930 0x7f774c0abdf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f775c104cd0 0x7f775c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f775c103140 0x7f775c199070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 --2- 172.21.15.64:0/1321883620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104330 0x7f775c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 >> 172.21.15.64:0/1321883620 conn(0x7f775c0fe6b0 msgr2=0x7f775c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 shutdown_connections 2024-05-08T21:55:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:04.411+0000 7f7762c38700 1 -- 172.21.15.64:0/1321883620 wait complete. 2024-05-08T21:55:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:05 smithi186 ceph-mon[46619]: pgmap v273: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 111 MiB/s wr, 1.26k op/s 2024-05-08T21:55:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:05 smithi064 ceph-mon[42500]: pgmap v273: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 111 MiB/s wr, 1.26k op/s 2024-05-08T21:55:06.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:05 smithi160 ceph-mon[46962]: pgmap v273: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 111 MiB/s wr, 1.26k op/s 2024-05-08T21:55:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 15.0 s, 4980.2 tps, lat 20.285 ms stddev 23.517 2024-05-08T21:55:08.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:07 smithi186 ceph-mon[46619]: pgmap v274: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 94 MiB/s wr, 1.20k op/s 2024-05-08T21:55:08.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:07 smithi064 ceph-mon[42500]: pgmap v274: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 94 MiB/s wr, 1.20k op/s 2024-05-08T21:55:08.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:07 smithi160 ceph-mon[46962]: pgmap v274: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 94 MiB/s wr, 1.20k op/s 2024-05-08T21:55:10.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:09 smithi186 ceph-mon[46619]: pgmap v275: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 89 MiB/s wr, 1.19k op/s 2024-05-08T21:55:10.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:09 smithi064 ceph-mon[42500]: pgmap v275: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 89 MiB/s wr, 1.19k op/s 2024-05-08T21:55:10.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:09 smithi160 ceph-mon[46962]: pgmap v275: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 89 MiB/s wr, 1.19k op/s 2024-05-08T21:55:11.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 20.0 s, 3875.6 tps, lat 25.422 ms stddev 31.276 2024-05-08T21:55:12.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:11 smithi186 ceph-mon[46619]: pgmap v276: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 82 MiB/s wr, 1.25k op/s 2024-05-08T21:55:12.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:11 smithi064 ceph-mon[42500]: pgmap v276: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 82 MiB/s wr, 1.25k op/s 2024-05-08T21:55:12.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:11 smithi160 ceph-mon[46962]: pgmap v276: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 82 MiB/s wr, 1.25k op/s 2024-05-08T21:55:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:13 smithi186 ceph-mon[46619]: pgmap v277: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 MiB/s wr, 1.13k op/s 2024-05-08T21:55:14.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:13 smithi064 ceph-mon[42500]: pgmap v277: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 MiB/s wr, 1.13k op/s 2024-05-08T21:55:14.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:13 smithi160 ceph-mon[46962]: pgmap v277: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 MiB/s wr, 1.13k op/s 2024-05-08T21:55:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:15 smithi064 ceph-mon[42500]: pgmap v278: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 MiB/s wr, 1.20k op/s 2024-05-08T21:55:16.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:15 smithi160 ceph-mon[46962]: pgmap v278: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 MiB/s wr, 1.20k op/s 2024-05-08T21:55:16.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:15 smithi186 ceph-mon[46619]: pgmap v278: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 MiB/s wr, 1.20k op/s 2024-05-08T21:55:16.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 25.0 s, 3436.6 tps, lat 29.492 ms stddev 45.388 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 -- 172.21.15.64:0/518378297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 msgr2=0x7fc93c10a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 --2- 172.21.15.64:0/518378297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c10a3d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0098f0 tx=0x7fc92c009c00 comp rx=0 tx=0).stop 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 -- 172.21.15.64:0/518378297 shutdown_connections 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 --2- 172.21.15.64:0/518378297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c10a3d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 --2- 172.21.15.64:0/518378297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10d2e0 0x7fc93c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 --2- 172.21.15.64:0/518378297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10a910 0x7fc93c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.604+0000 7fc941a76700 1 -- 172.21.15.64:0/518378297 >> 172.21.15.64:0/518378297 conn(0x7fc93c06df30 msgr2=0x7fc93c070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.605+0000 7fc941a76700 1 -- 172.21.15.64:0/518378297 shutdown_connections 2024-05-08T21:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.608+0000 7fc941a76700 1 -- 172.21.15.64:0/518378297 wait complete. 2024-05-08T21:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.608+0000 7fc941a76700 1 Processor -- start 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.608+0000 7fc941a76700 1 -- start start 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc941a76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc941a76700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10a910 0x7fc93c1a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc941a76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10d2e0 0x7fc93c1ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc941a76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c1ad8e0 con 0x7fc93c10d2e0 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc941a76700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c1ada20 con 0x7fc93c107fa0 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc941a76700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c1adbb0 con 0x7fc93c10a910 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c1a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:37546/0 (socket says 172.21.15.64:37546) 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 -- 172.21.15.64:0/928163553 learned_addr learned my addr 172.21.15.64:0/928163553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 -- 172.21.15.64:0/928163553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10a910 msgr2=0x7fc93c1a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93affd700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10a910 0x7fc93c1a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10a910 0x7fc93c1a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 -- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10d2e0 msgr2=0x7fc93c1ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93bfff700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10d2e0 0x7fc93c1ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.609+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10d2e0 0x7fc93c1ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc93b7fe700 1 -- 172.21.15.64:0/928163553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009520 con 0x7fc93c107fa0 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc93affd700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10a910 0x7fc93c1a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc93bfff700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10d2e0 0x7fc93c1ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c1a5db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009230 tx=0x7fc92c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc938ff9700 1 -- 172.21.15.64:0/928163553 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c01c070 con 0x7fc93c107fa0 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc938ff9700 1 -- 172.21.15.64:0/928163553 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c0054f0 con 0x7fc93c107fa0 2024-05-08T21:55:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc938ff9700 1 -- 172.21.15.64:0/928163553 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c020410 con 0x7fc93c107fa0 2024-05-08T21:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c1ade30 con 0x7fc93c107fa0 2024-05-08T21:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.610+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c1ae1a0 con 0x7fc93c107fa0 2024-05-08T21:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.614+0000 7fc938ff9700 1 -- 172.21.15.64:0/928163553 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc92c005030 con 0x7fc93c107fa0 2024-05-08T21:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 Processor -- start 2024-05-08T21:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc938ff9700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9240a11f0 0x7fc9240a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc938ff9700 1 -- 172.21.15.64:0/928163553 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc92c0cdaf0 con 0x7fc93c107fa0 2024-05-08T21:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 -- start start 2024-05-08T21:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc93bfff700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9240a11f0 0x7fc9240a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19c9eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19c9fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19ca02d0 0x558b19ca0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19bf60f0 con 0x558b19c9f3e0 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19bf3ab0 con 0x558b19c9ea80 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.615+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19bef3a0 con 0x558b19ca02d0 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19c9eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19c9eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:37548/0 (socket says 172.21.15.64:37548) 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 -- 172.21.15.64:0/208187710 learned_addr learned my addr 172.21.15.64:0/208187710 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93bfff700 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19ca02d0 0x558b19ca0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 -- 172.21.15.64:0/208187710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19ca02d0 msgr2=0x558b19ca0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19ca02d0 0x558b19ca0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 -- 172.21.15.64:0/208187710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 msgr2=0x558b19c9fcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19c9fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93b7fe700 1 -- 172.21.15.64:0/208187710 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b19a70350 con 0x558b19c9ea80 2024-05-08T21:55:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.616+0000 7fc93bfff700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9240a11f0 0x7fc9240a36b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc934007fd0 tx=0x7fc934013ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.617+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19c9eea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc92c09ace0 tx=0x7fc92c004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a35233d3c64aea5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.617+0000 7fc921ffb700 1 -- 172.21.15.64:0/208187710 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0a10c0 con 0x558b19c9ea80 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.617+0000 7fc921ffb700 1 -- 172.21.15.64:0/208187710 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c09afe0 con 0x558b19c9ea80 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.617+0000 7fc921ffb700 1 -- 172.21.15.64:0/208187710 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c09b240 con 0x558b19c9ea80 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 -- 172.21.15.64:0/208187710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 msgr2=0x558b19c9eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19c9eea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc92c09ace0 tx=0x7fc92c004990 comp rx=0 tx=0).stop 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 -- 172.21.15.64:0/208187710 shutdown_connections 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19c9eea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19ca02d0 0x558b19ca0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/208187710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19c9fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.618+0000 7fc94dc30b80 1 -- 172.21.15.64:0/208187710 >> 172.21.15.64:0/208187710 conn(0x558b19bfd1b0 msgr2=0x558b19c9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.619+0000 7fc94dc30b80 1 -- 172.21.15.64:0/208187710 shutdown_connections 2024-05-08T21:55:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.619+0000 7fc94dc30b80 1 -- 172.21.15.64:0/208187710 wait complete. 2024-05-08T21:55:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.619+0000 7fc94dc30b80 1 -- start start 2024-05-08T21:55:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 Processor -- start 2024-05-08T21:55:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19ca8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19ca8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19cafaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19b8b500 con 0x558b19c9f3e0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19b8b1a0 con 0x558b19ca02d0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19a72640 con 0x558b19c9ea80 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc94dc30b80 10 client.0 _ll_register_callbacks cb 0x558b19b94ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19cafaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19cafaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:37554/0 (socket says 172.21.15.64:37554) 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 -- 172.21.15.64:0/339225559 learned_addr learned my addr 172.21.15.64:0/339225559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93affd700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19ca8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19ca8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 -- 172.21.15.64:0/339225559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 msgr2=0x558b19ca8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19ca8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 -- 172.21.15.64:0/339225559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 msgr2=0x558b19ca8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19ca8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93bfff700 1 -- 172.21.15.64:0/339225559 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c1ae1a0 con 0x558b19ca02d0 2024-05-08T21:55:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.620+0000 7fc93affd700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19ca8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc93bfff700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19cafaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc934010a50 tx=0x7fc934004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94aa581724539faf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc91affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc91affd700 1 -- 172.21.15.64:0/339225559 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934003ca0 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b19b48210 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc91affd700 1 -- 172.21.15.64:0/339225559 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc934003e00 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc91affd700 1 -- 172.21.15.64:0/339225559 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc93402d030 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 10 client.25088 fetch_fsmap learned FSMap version 31 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558b19a1cb00 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc91affd700 1 -- 172.21.15.64:0/339225559 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b19b48210 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc91affd700 1 -- 172.21.15.64:0/339225559 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc934004420 con 0x558b19ca02d0 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 10 client.25088 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 10 client.25088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 20 client.25088 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19cc5490 0x558b19cc78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 4 client.25088 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:55:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.621+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b19c6e7b0 con 0x558b19cc5490 2024-05-08T21:55:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.622+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19cc5490 0x558b19cc78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.624+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19cc5490 0x558b19cc78b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f73e9f3e83bebb server_cookie=f0998faea0ed9cdf in_seq=0 out_seq=0 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.624+0000 7fc91affd700 10 client.25088 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc91affd700 1 -- 172.21.15.64:0/339225559 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b19c6e7b0 con 0x558b19cc5490 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc91affd700 10 client.25088 handle_command_reply: tid=0 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 client.25088 shutdown 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 msgr2=0x558b19cafaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19cafaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc934010a50 tx=0x7fc934004af0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 shutdown_connections 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19cafaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19cc5490 0x558b19cc78b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19ca8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc91affd700 0 client.25088 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/339225559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19ca8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.625+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 >> 172.21.15.64:0/339225559 conn(0x558b19bfd1b0 msgr2=0x558b19c99860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.626+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 shutdown_connections 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.626+0000 7fc94dc30b80 1 -- 172.21.15.64:0/339225559 wait complete. 2024-05-08T21:55:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.626+0000 7fc94dc30b80 20 client.25088 trim_cache size 0 max 16384 2024-05-08T21:55:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 Processor -- start 2024-05-08T21:55:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 -- start start 2024-05-08T21:55:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19c9ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19cafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19ca06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19c94360 con 0x558b19c9f3e0 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19cc1cb0 con 0x558b19ca02d0 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.648+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19c9a930 con 0x558b19c9ea80 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19c9ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19c9ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:60066/0 (socket says 172.21.15.64:60066) 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 -- 172.21.15.64:0/2730210467 learned_addr learned my addr 172.21.15.64:0/2730210467 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93bfff700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19ca06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 -- 172.21.15.64:0/2730210467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 msgr2=0x558b19ca06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19ca06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 -- 172.21.15.64:0/2730210467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 msgr2=0x558b19cafab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93affd700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19cafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19cafab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 -- 172.21.15.64:0/2730210467 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b19bb5500 con 0x558b19c9ea80 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93bfff700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19ca06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93affd700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19cafab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.649+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19c9ee60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc92c097a50 tx=0x7fc92c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2520739356864adf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.650+0000 7fc9227fc700 1 -- 172.21.15.64:0/2730210467 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0a10b0 con 0x558b19c9ea80 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.650+0000 7fc9227fc700 1 -- 172.21.15.64:0/2730210467 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c099400 con 0x558b19c9ea80 2024-05-08T21:55:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.650+0000 7fc9227fc700 1 -- 172.21.15.64:0/2730210467 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c09a4d0 con 0x558b19c9ea80 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.650+0000 7fc94dc30b80 1 -- 172.21.15.64:0/2730210467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 msgr2=0x558b19c9ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.650+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19c9ee60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc92c097a50 tx=0x7fc92c009c00 comp rx=0 tx=0).stop 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- 172.21.15.64:0/2730210467 shutdown_connections 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19ca02d0 0x558b19ca06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9ea80 0x558b19c9ee60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/2730210467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19c9f3e0 0x558b19cafab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- 172.21.15.64:0/2730210467 >> 172.21.15.64:0/2730210467 conn(0x558b19bfd1b0 msgr2=0x558b19ca6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- 172.21.15.64:0/2730210467 shutdown_connections 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- 172.21.15.64:0/2730210467 wait complete. 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- start start 2024-05-08T21:55:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 Processor -- start 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19cad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 0x558b19cad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19ca02d0 0x558b19cae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19cae980 con 0x558b19ca02d0 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19caeaf0 con 0x558b19c9ea80 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19caecb0 con 0x558b19c9f3e0 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.651+0000 7fc94dc30b80 10 client.0 _ll_register_callbacks cb 0x558b19ca01d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 0x558b19cad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 0x558b19cad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:60070/0 (socket says 172.21.15.64:60070) 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19ca02d0 0x558b19cae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 -- 172.21.15.64:0/142971924 learned_addr learned my addr 172.21.15.64:0/142971924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19cad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 -- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 msgr2=0x558b19cad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19cad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 -- 172.21.15.64:0/142971924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19ca02d0 msgr2=0x558b19cae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19ca02d0 0x558b19cae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 -- 172.21.15.64:0/142971924 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b19a1cb00 con 0x558b19c9f3e0 2024-05-08T21:55:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.652+0000 7fc93affd700 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 0x558b19cad570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9300116b0 tx=0x7fc9300119c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ceb2743b1139229 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc91affd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc91affd700 1 -- 172.21.15.64:0/142971924 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930004440 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc91affd700 1 -- 172.21.15.64:0/142971924 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc930003680 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc92c09afe0 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc91affd700 1 -- 172.21.15.64:0/142971924 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc930021030 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc94dc30b80 10 client.25036 fetch_fsmap learned FSMap version 31 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558b19a70350 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc91affd700 1 -- 172.21.15.64:0/142971924 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c09afe0 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.654+0000 7fc91affd700 1 -- 172.21.15.64:0/142971924 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc930012820 con 0x558b19c9f3e0 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.655+0000 7fc94dc30b80 10 client.25036 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.655+0000 7fc94dc30b80 10 client.25036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.655+0000 7fc94dc30b80 20 client.25036 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.655+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19d1c0e0 0x558b19d1e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.655+0000 7fc94dc30b80 4 client.25036 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:55:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.655+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b19be7e60 con 0x558b19d1c0e0 2024-05-08T21:55:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.657+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19d1c0e0 0x558b19d1e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.658+0000 7fc93b7fe700 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19d1c0e0 0x558b19d1e500 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515f69e058c3d676 server_cookie=e302bcdc057221b in_seq=0 out_seq=0 2024-05-08T21:55:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.658+0000 7fc91affd700 10 client.25036 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.675+0000 7fc91affd700 1 -- 172.21.15.64:0/142971924 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b19be7e60 con 0x558b19d1c0e0 2024-05-08T21:55:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.675+0000 7fc91affd700 10 client.25036 handle_command_reply: tid=0 2024-05-08T21:55:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.675+0000 7fc94dc30b80 1 client.25036 shutdown 2024-05-08T21:55:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.675+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 msgr2=0x558b19cad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.675+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 0x558b19cad570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9300116b0 tx=0x7fc9300119c0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 shutdown_connections 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b19c9ea80 0x558b19cad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x558b19d1c0e0 0x558b19d1e500 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b19ca02d0 0x558b19cae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 --2- 172.21.15.64:0/142971924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b19c9f3e0 0x558b19cad570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 >> 172.21.15.64:0/142971924 conn(0x558b19bfd1b0 msgr2=0x558b19ca3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc91affd700 0 client.25036 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 shutdown_connections 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 1 -- 172.21.15.64:0/142971924 wait complete. 2024-05-08T21:55:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.676+0000 7fc94dc30b80 20 client.25036 trim_cache size 0 max 16384 2024-05-08T21:55:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:55:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:55:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:55:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9240a11f0 msgr2=0x7fc9240a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9240a11f0 0x7fc9240a36b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc934007fd0 tx=0x7fc934013ab0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 msgr2=0x7fc93c1a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c1a5db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009230 tx=0x7fc92c00b8f0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 shutdown_connections 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fc9240a11f0 0x7fc9240a36b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc93c107fa0 0x7fc93c1a5db0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c10d2e0 0x7fc93c1ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 --2- 172.21.15.64:0/928163553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc93c10a910 0x7fc93c1a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 >> 172.21.15.64:0/928163553 conn(0x7fc93c06df30 msgr2=0x7fc93c10ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 shutdown_connections 2024-05-08T21:55:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:17.677+0000 7fc941a76700 1 -- 172.21.15.64:0/928163553 wait complete. 2024-05-08T21:55:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:17 smithi064 ceph-mon[42500]: pgmap v279: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 1.27k op/s 2024-05-08T21:55:18.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:17 smithi160 ceph-mon[46962]: pgmap v279: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 1.27k op/s 2024-05-08T21:55:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:17 smithi186 ceph-mon[46619]: pgmap v279: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 1.27k op/s 2024-05-08T21:55:19.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:18 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:19.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:18 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:19.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:18 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:20.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:19 smithi064 ceph-mon[42500]: pgmap v280: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 65 MiB/s wr, 1.78k op/s 2024-05-08T21:55:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:19 smithi160 ceph-mon[46962]: pgmap v280: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 65 MiB/s wr, 1.78k op/s 2024-05-08T21:55:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:19 smithi186 ceph-mon[46619]: pgmap v280: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 65 MiB/s wr, 1.78k op/s 2024-05-08T21:55:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 30.0 s, 4589.8 tps, lat 21.817 ms stddev 23.290 2024-05-08T21:55:22.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:21 smithi064 ceph-mon[42500]: pgmap v281: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 78 MiB/s wr, 2.51k op/s 2024-05-08T21:55:22.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:21 smithi160 ceph-mon[46962]: pgmap v281: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 78 MiB/s wr, 2.51k op/s 2024-05-08T21:55:22.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:21 smithi186 ceph-mon[46619]: pgmap v281: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 78 MiB/s wr, 2.51k op/s 2024-05-08T21:55:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:23 smithi064 ceph-mon[42500]: pgmap v282: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 77 MiB/s wr, 2.47k op/s 2024-05-08T21:55:24.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:23 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:55:24.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:23 smithi160 ceph-mon[46962]: pgmap v282: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 77 MiB/s wr, 2.47k op/s 2024-05-08T21:55:24.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:23 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:55:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:23 smithi186 ceph-mon[46619]: pgmap v282: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 77 MiB/s wr, 2.47k op/s 2024-05-08T21:55:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:23 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:55:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: pgmap v283: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 74 MiB/s wr, 2.48k op/s 2024-05-08T21:55:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:25 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: pgmap v283: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 74 MiB/s wr, 2.48k op/s 2024-05-08T21:55:25.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:25 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: pgmap v283: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 74 MiB/s wr, 2.48k op/s 2024-05-08T21:55:25.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:25.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:25 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 35.0 s, 4284.2 tps, lat 23.320 ms stddev 27.018 2024-05-08T21:55:26.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:26 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:55:26.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:26 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:55:26.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:26 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:26.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:26 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:55:26.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:26 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:55:26.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:26 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:26 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:55:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:26 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:55:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:26 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:55:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:27 smithi160 ceph-mon[46962]: pgmap v284: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 MiB/s wr, 2.28k op/s 2024-05-08T21:55:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:27 smithi186 ceph-mon[46619]: pgmap v284: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 MiB/s wr, 2.28k op/s 2024-05-08T21:55:28.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:27 smithi064 ceph-mon[42500]: pgmap v284: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 MiB/s wr, 2.28k op/s 2024-05-08T21:55:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:28 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:28 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:29.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:28 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:55:30.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:29 smithi160 ceph-mon[46962]: pgmap v285: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 56 MiB/s wr, 2.17k op/s 2024-05-08T21:55:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:29 smithi186 ceph-mon[46619]: pgmap v285: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 56 MiB/s wr, 2.17k op/s 2024-05-08T21:55:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:29 smithi064 ceph-mon[42500]: pgmap v285: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 56 MiB/s wr, 2.17k op/s 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 -- 172.21.15.64:0/3148893665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108101990 msgr2=0x7f1108108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 --2- 172.21.15.64:0/3148893665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108101990 0x7f1108108240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1100009fc0 tx=0x7f110000b330 comp rx=0 tx=0).stop 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 -- 172.21.15.64:0/3148893665 shutdown_connections 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 --2- 172.21.15.64:0/3148893665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108101990 0x7f1108108240 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 --2- 172.21.15.64:0/3148893665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11080ffe00 0x7f1108100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 --2- 172.21.15.64:0/3148893665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1108100ff0 0x7f1108101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.850+0000 7f110c92c700 1 -- 172.21.15.64:0/3148893665 >> 172.21.15.64:0/3148893665 conn(0x7f11080fb390 msgr2=0x7f11080fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.851+0000 7f110c92c700 1 -- 172.21.15.64:0/3148893665 shutdown_connections 2024-05-08T21:55:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.851+0000 7f110c92c700 1 -- 172.21.15.64:0/3148893665 wait complete. 2024-05-08T21:55:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.851+0000 7f110c92c700 1 Processor -- start 2024-05-08T21:55:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 -- start start 2024-05-08T21:55:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11080ffe00 0x7f1108194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 0x7f11081952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1108101990 0x7f110819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110819c8e0 con 0x7f1108100ff0 2024-05-08T21:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110806a9c0 con 0x7f11080ffe00 2024-05-08T21:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110c92c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110806ab50 con 0x7f1108101990 2024-05-08T21:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 0x7f11081952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 0x7f11081952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59704/0 (socket says 172.21.15.64:59704) 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 -- 172.21.15.64:0/2074778856 learned_addr learned my addr 172.21.15.64:0/2074778856 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110659c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11080ffe00 0x7f1108194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 -- 172.21.15.64:0/2074778856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1108101990 msgr2=0x7f110819c310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1108101990 0x7f110819c310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 -- 172.21.15.64:0/2074778856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11080ffe00 msgr2=0x7f1108194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11080ffe00 0x7f1108194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f1105d9b700 1 -- 172.21.15.64:0/2074778856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100009c40 con 0x7f1108100ff0 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.852+0000 7f110659c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11080ffe00 0x7f1108194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.853+0000 7f1105d9b700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 0x7f11081952f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097d0 tx=0x7f10fc009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.853+0000 7f10f77fe700 1 -- 172.21.15.64:0/2074778856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc010040 con 0x7f1108100ff0 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.853+0000 7f10f77fe700 1 -- 172.21.15.64:0/2074778856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10fc014ad0 con 0x7f1108100ff0 2024-05-08T21:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.853+0000 7f10f77fe700 1 -- 172.21.15.64:0/2074778856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc013b50 con 0x7f1108100ff0 2024-05-08T21:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.854+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110806ae30 con 0x7f1108100ff0 2024-05-08T21:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.854+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110806b2f0 con 0x7f1108100ff0 2024-05-08T21:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.857+0000 7f10f77fe700 1 -- 172.21.15.64:0/2074778856 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f10fc014c40 con 0x7f1108100ff0 2024-05-08T21:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.858+0000 7f10f77fe700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f10f00a5680 0x7f10f00a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.858+0000 7f10f77fe700 1 -- 172.21.15.64:0/2074778856 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10fc0cc3f0 con 0x7f1108100ff0 2024-05-08T21:55:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.858+0000 7f1118ae6b80 1 Processor -- start 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 -- start start 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 0x55f96546cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546d3e0 0x55f96546dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96546e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9653c40f0 con 0x55f96546e2d0 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9653c1ab0 con 0x55f96546ca80 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9653bd3a0 con 0x55f96546d3e0 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 0x55f96546cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546d3e0 0x55f96546dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 0x55f96546cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54084/0 (socket says 172.21.15.64:54084) 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 -- 172.21.15.64:0/1594247535 learned_addr learned my addr 172.21.15.64:0/1594247535 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 -- 172.21.15.64:0/1594247535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546d3e0 msgr2=0x55f96546dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546d3e0 0x55f96546dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 -- 172.21.15.64:0/1594247535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 msgr2=0x55f96546e750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96546e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.860+0000 7f110659c700 1 -- 172.21.15.64:0/1594247535 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f96523e350 con 0x55f96546ca80 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.861+0000 7f110659c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f10f00a5680 0x7f10f00a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.861+0000 7f110659c700 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 0x55f96546cea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f10f8011760 tx=0x7f10f8011a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6207b5734815f90a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.861+0000 7f10f4ff9700 1 -- 172.21.15.64:0/1594247535 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8021030 con 0x55f96546ca80 2024-05-08T21:55:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.861+0000 7f110659c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f10f00a5680 0x7f10f00a7b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f10f8021000 tx=0x7f10f801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.861+0000 7f10f4ff9700 1 -- 172.21.15.64:0/1594247535 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10f801c940 con 0x55f96546ca80 2024-05-08T21:55:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.861+0000 7f10f4ff9700 1 -- 172.21.15.64:0/1594247535 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8016070 con 0x55f96546ca80 2024-05-08T21:55:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.862+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1594247535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 msgr2=0x55f96546cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.862+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 0x55f96546cea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f10f8011760 tx=0x7f10f8011a70 comp rx=0 tx=0).stop 2024-05-08T21:55:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1594247535 shutdown_connections 2024-05-08T21:55:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546ca80 0x55f96546cea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96546e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1594247535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546d3e0 0x55f96546dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1594247535 >> 172.21.15.64:0/1594247535 conn(0x55f9653cb1b0 msgr2=0x55f965469440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1594247535 shutdown_connections 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1594247535 wait complete. 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.863+0000 7f1118ae6b80 1 -- start start 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 Processor -- start 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f9654768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f965476e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f965359500 con 0x55f96546e2d0 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9653591a0 con 0x55f96546d3e0 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f965240640 con 0x55f96546ca80 2024-05-08T21:55:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1118ae6b80 10 client.0 _ll_register_callbacks cb 0x55f965362ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f965476e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1106d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f965476e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54096/0 (socket says 172.21.15.64:54096) 2024-05-08T21:55:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f110659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f9654768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.864+0000 7f1105d9b700 1 -- 172.21.15.64:0/1115496056 learned_addr learned my addr 172.21.15.64:0/1115496056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f110659c700 1 -- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 msgr2=0x55f965476e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f110659c700 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f965476e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f110659c700 1 -- 172.21.15.64:0/1115496056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 msgr2=0x55f96547de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f110659c700 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f110659c700 1 -- 172.21.15.64:0/1115496056 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f8011440 con 0x55f96546ca80 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f1106d9d700 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f110659c700 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f9654768f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009560 tx=0x7f10f801cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c3c261c111c31d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f10edffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f10edffb700 1 -- 172.21.15.64:0/1115496056 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8016070 con 0x55f96546ca80 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f965316210 con 0x55f96546ca80 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.865+0000 7f10edffb700 1 -- 172.21.15.64:0/1115496056 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10f8022070 con 0x55f96546ca80 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.866+0000 7f10edffb700 1 -- 172.21.15.64:0/1115496056 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10f8005250 con 0x55f96546ca80 2024-05-08T21:55:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.866+0000 7f1118ae6b80 10 client.25045 fetch_fsmap learned FSMap version 33 2024-05-08T21:55:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.866+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f9651eab00 con 0x55f96546ca80 2024-05-08T21:55:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f10edffb700 1 -- 172.21.15.64:0/1115496056 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f965316210 con 0x55f96546ca80 2024-05-08T21:55:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f10edffb700 1 -- 172.21.15.64:0/1115496056 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f10f8004710 con 0x55f96546ca80 2024-05-08T21:55:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f1118ae6b80 10 client.25045 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f1118ae6b80 10 client.25045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f1118ae6b80 20 client.25045 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f965482620 0x55f965484a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f1118ae6b80 4 client.25045 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.867+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f96543c7b0 con 0x55f965482620 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.868+0000 7f1105d9b700 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f965482620 0x55f965484a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.868+0000 7f1105d9b700 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f965482620 0x55f965484a40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6320a051d979b9fb server_cookie=3f3e067e0541603d in_seq=0 out_seq=0 2024-05-08T21:55:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.868+0000 7f10edffb700 10 client.25045 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f10edffb700 1 -- 172.21.15.64:0/1115496056 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f96543c7b0 con 0x55f965482620 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f10edffb700 10 client.25045 handle_command_reply: tid=0 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 client.25045 shutdown 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 msgr2=0x55f9654768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f9654768f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009560 tx=0x7f10f801cce0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 shutdown_connections 2024-05-08T21:55:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f965482620 0x55f965484a40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f9654768f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f10edffb700 0 client.25045 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1115496056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f965476e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.869+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 >> 172.21.15.64:0/1115496056 conn(0x55f9653cb1b0 msgr2=0x55f965467860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.870+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 shutdown_connections 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.870+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1115496056 wait complete. 2024-05-08T21:55:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.870+0000 7f1118ae6b80 20 client.25045 trim_cache size 0 max 16384 2024-05-08T21:55:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.894+0000 7f1118ae6b80 1 Processor -- start 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 -- start start 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546d3e0 0x55f965475e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546e2d0 0x55f96547de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9654627c0 con 0x55f96546d3e0 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f965475170 con 0x55f96546e2d0 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96547eff0 con 0x55f96546ca80 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546d3e0 0x55f965475e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:50634/0 (socket says 172.21.15.64:50634) 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 -- 172.21.15.64:0/3523052353 learned_addr learned my addr 172.21.15.64:0/3523052353 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f1106d9d700 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546e2d0 0x55f96547de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 -- 172.21.15.64:0/3523052353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546e2d0 msgr2=0x55f96547de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546e2d0 0x55f96547de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 -- 172.21.15.64:0/3523052353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546d3e0 msgr2=0x55f965475e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546d3e0 0x55f965475e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.895+0000 7f110659c700 1 -- 172.21.15.64:0/3523052353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f965383500 con 0x55f96546ca80 2024-05-08T21:55:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.896+0000 7f110659c700 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547e520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005010 tx=0x7f10f801c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3debe93f1e8dadac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.896+0000 7f10f57fa700 1 -- 172.21.15.64:0/3523052353 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8016070 con 0x55f96546ca80 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.896+0000 7f10f57fa700 1 -- 172.21.15.64:0/3523052353 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10f8017040 con 0x55f96546ca80 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.896+0000 7f10f57fa700 1 -- 172.21.15.64:0/3523052353 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f80048f0 con 0x55f96546ca80 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 -- 172.21.15.64:0/3523052353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 msgr2=0x55f96547e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547e520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005010 tx=0x7f10f801c860 comp rx=0 tx=0).stop 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 -- 172.21.15.64:0/3523052353 shutdown_connections 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546e2d0 0x55f96547de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547e520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/3523052353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546d3e0 0x55f965475e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 -- 172.21.15.64:0/3523052353 >> 172.21.15.64:0/3523052353 conn(0x55f9653cb1b0 msgr2=0x55f965474890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 -- 172.21.15.64:0/3523052353 shutdown_connections 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 -- 172.21.15.64:0/3523052353 wait complete. 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 -- start start 2024-05-08T21:55:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.897+0000 7f1118ae6b80 1 Processor -- start 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f96547b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96547cbd0 con 0x55f96546e2d0 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96547cd40 con 0x55f96546d3e0 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96547cf00 con 0x55f96546ca80 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1118ae6b80 10 client.0 _ll_register_callbacks cb 0x55f965362ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f96547b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f96547b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54118/0 (socket says 172.21.15.64:54118) 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 -- 172.21.15.64:0/1546317995 learned_addr learned my addr 172.21.15.64:0/1546317995 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f110659c700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1106d9d700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 -- 172.21.15.64:0/1546317995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 msgr2=0x55f96547b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 -- 172.21.15.64:0/1546317995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 msgr2=0x55f96547c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.898+0000 7f1105d9b700 1 -- 172.21.15.64:0/1546317995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9651eab00 con 0x55f96546d3e0 2024-05-08T21:55:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f10edffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f1105d9b700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f96547b680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0ca270 tx=0x7f10fc0b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0ba2387b633dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f10edffb700 1 -- 172.21.15.64:0/1546317995 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc09d680 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f10edffb700 1 -- 172.21.15.64:0/1546317995 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10fc0b7e40 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10f801c940 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f10edffb700 1 -- 172.21.15.64:0/1546317995 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10fc0a1cd0 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f1118ae6b80 10 client.25112 fetch_fsmap learned FSMap version 33 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f965242c70 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.899+0000 7f10edffb700 1 -- 172.21.15.64:0/1546317995 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10f801c940 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f10edffb700 1 -- 172.21.15.64:0/1546317995 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f10fc0b90b0 con 0x55f96546d3e0 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f1118ae6b80 10 client.25112 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f1118ae6b80 10 client.25112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f1118ae6b80 20 client.25112 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f9654e18d0 0x55f9654e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f1118ae6b80 4 client.25112 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:55:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9653b5e60 con 0x55f9654e18d0 2024-05-08T21:55:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f110659c700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f9654e18d0 0x55f9654e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f10edffb700 10 client.25112 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.900+0000 7f110659c700 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f9654e18d0 0x55f9654e3cf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7008013a7b31148 server_cookie=2cdef1060dc86f0f in_seq=0 out_seq=0 2024-05-08T21:55:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.950+0000 7f10edffb700 1 -- 172.21.15.64:0/1546317995 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9653b5e60 con 0x55f9654e18d0 2024-05-08T21:55:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.950+0000 7f10edffb700 10 client.25112 handle_command_reply: tid=0 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.950+0000 7f1118ae6b80 1 client.25112 shutdown 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 msgr2=0x55f96547b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f96547b680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0ca270 tx=0x7f10fc0b9040 comp rx=0 tx=0).stop 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 shutdown_connections 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f9654e18d0 0x55f9654e3cf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f96546e2d0 0x55f96547c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f96546ca80 0x55f96547b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 --2- 172.21.15.64:0/1546317995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f96546d3e0 0x55f96547b680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 >> 172.21.15.64:0/1546317995 conn(0x55f9653cb1b0 msgr2=0x55f965475e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f10edffb700 0 client.25112 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 shutdown_connections 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 1 -- 172.21.15.64:0/1546317995 wait complete. 2024-05-08T21:55:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.951+0000 7f1118ae6b80 20 client.25112 trim_cache size 0 max 16384 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f10f00a5680 msgr2=0x7f10f00a7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f10f00a5680 0x7f10f00a7b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f10f8021000 tx=0x7f10f801e000 comp rx=0 tx=0).stop 2024-05-08T21:55:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 msgr2=0x7f11081952f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 0x7f11081952f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097d0 tx=0x7f10fc009b90 comp rx=0 tx=0).stop 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 shutdown_connections 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f10f00a5680 0x7f10f00a7b40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11080ffe00 0x7f1108194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1108101990 0x7f110819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 --2- 172.21.15.64:0/2074778856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108100ff0 0x7f11081952f0 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097d0 tx=0x7f10fc009b90 comp rx=0 tx=0).stop 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.952+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 >> 172.21.15.64:0/2074778856 conn(0x7f11080fb390 msgr2=0x7f11081042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.954+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 shutdown_connections 2024-05-08T21:55:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:30.954+0000 7f110c92c700 1 -- 172.21.15.64:0/2074778856 wait complete. 2024-05-08T21:55:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 40.0 s, 3897.3 tps, lat 25.609 ms stddev 29.590 2024-05-08T21:55:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:31 smithi160 ceph-mon[46962]: pgmap v286: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 MiB/s wr, 1.82k op/s 2024-05-08T21:55:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:31 smithi186 ceph-mon[46619]: pgmap v286: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 MiB/s wr, 1.82k op/s 2024-05-08T21:55:32.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:31 smithi064 ceph-mon[42500]: pgmap v286: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 MiB/s wr, 1.82k op/s 2024-05-08T21:55:34.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:33 smithi160 ceph-mon[46962]: pgmap v287: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 1.18k op/s 2024-05-08T21:55:34.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:33 smithi186 ceph-mon[46619]: pgmap v287: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 1.18k op/s 2024-05-08T21:55:34.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:33 smithi064 ceph-mon[42500]: pgmap v287: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 1.18k op/s 2024-05-08T21:55:36.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:35 smithi160 ceph-mon[46962]: pgmap v288: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 MiB/s wr, 1.28k op/s 2024-05-08T21:55:36.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:35 smithi186 ceph-mon[46619]: pgmap v288: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 MiB/s wr, 1.28k op/s 2024-05-08T21:55:36.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:35 smithi064 ceph-mon[42500]: pgmap v288: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 MiB/s wr, 1.28k op/s 2024-05-08T21:55:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 45.0 s, 3723.7 tps, lat 26.905 ms stddev 51.554 2024-05-08T21:55:38.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:37 smithi160 ceph-mon[46962]: pgmap v289: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.29k op/s 2024-05-08T21:55:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:37 smithi186 ceph-mon[46619]: pgmap v289: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.29k op/s 2024-05-08T21:55:38.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:37 smithi064 ceph-mon[42500]: pgmap v289: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.29k op/s 2024-05-08T21:55:40.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:39 smithi160 ceph-mon[46962]: pgmap v290: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 43 MiB/s wr, 1.70k op/s 2024-05-08T21:55:40.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:39 smithi186 ceph-mon[46619]: pgmap v290: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 43 MiB/s wr, 1.70k op/s 2024-05-08T21:55:40.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:39 smithi064 ceph-mon[42500]: pgmap v290: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 43 MiB/s wr, 1.70k op/s 2024-05-08T21:55:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 50.0 s, 4203.0 tps, lat 23.795 ms stddev 26.553 2024-05-08T21:55:42.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:41 smithi160 ceph-mon[46962]: pgmap v291: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 MiB/s wr, 2.35k op/s 2024-05-08T21:55:42.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:41 smithi186 ceph-mon[46619]: pgmap v291: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 MiB/s wr, 2.35k op/s 2024-05-08T21:55:42.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:41 smithi064 ceph-mon[42500]: pgmap v291: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 MiB/s wr, 2.35k op/s 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.123+0000 7f1c0b70f700 1 -- 172.21.15.64:0/199115719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c040ff1d0 msgr2=0x7f1c040ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.123+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/199115719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c040ff1d0 0x7f1c040ff5f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009990 tx=0x7f1c00009ca0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.124+0000 7f1c0b70f700 1 -- 172.21.15.64:0/199115719 shutdown_connections 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.124+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/199115719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c040ff1d0 0x7f1c040ff5f0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009990 tx=0x7f1c00009ca0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.124+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/199115719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c04102b40 0x7f1c041093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.124+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/199115719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c041021a0 0x7f1c04102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.124+0000 7f1c0b70f700 1 -- 172.21.15.64:0/199115719 >> 172.21.15.64:0/199115719 conn(0x7f1c040fab20 msgr2=0x7f1c040fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.125+0000 7f1c0b70f700 1 -- 172.21.15.64:0/199115719 shutdown_connections 2024-05-08T21:55:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.125+0000 7f1c0b70f700 1 -- 172.21.15.64:0/199115719 wait complete. 2024-05-08T21:55:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.126+0000 7f1c0b70f700 1 Processor -- start 2024-05-08T21:55:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 -- start start 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c041021a0 0x7f1c04106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c04102b40 0x7f1c04107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 0x7f1c04103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041043f0 con 0x7f1c04102b40 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04198fc0 con 0x7f1c041037f0 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c0b70f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04199150 con 0x7f1c041021a0 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 0x7f1c04103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 0x7f1c04103c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40162/0 (socket says 172.21.15.64:40162) 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 -- 172.21.15.64:0/1049125721 learned_addr learned my addr 172.21.15.64:0/1049125721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c094ab700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c041021a0 0x7f1c04106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 -- 172.21.15.64:0/1049125721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c041021a0 msgr2=0x7f1c04106ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c041021a0 0x7f1c04106ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 -- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c04102b40 msgr2=0x7f1c04107220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c04102b40 0x7f1c04107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.127+0000 7f1c09cac700 1 -- 172.21.15.64:0/1049125721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c000095f0 con 0x7f1c041037f0 2024-05-08T21:55:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.128+0000 7f1c09cac700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 0x7f1c04103c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00ef30 tx=0x7f1bfc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.128+0000 7f1bfa7fc700 1 -- 172.21.15.64:0/1049125721 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc018070 con 0x7f1c041037f0 2024-05-08T21:55:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.128+0000 7f1bfa7fc700 1 -- 172.21.15.64:0/1049125721 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc00f040 con 0x7f1c041037f0 2024-05-08T21:55:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.128+0000 7f1bfa7fc700 1 -- 172.21.15.64:0/1049125721 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00cdb0 con 0x7f1c041037f0 2024-05-08T21:55:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.129+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c041993d0 con 0x7f1c041037f0 2024-05-08T21:55:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.129+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c04199860 con 0x7f1c041037f0 2024-05-08T21:55:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.134+0000 7f1c178c9b80 1 Processor -- start 2024-05-08T21:55:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.134+0000 7f1bfa7fc700 1 -- 172.21.15.64:0/1049125721 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1bfc003a00 con 0x7f1c041037f0 2024-05-08T21:55:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 -- start start 2024-05-08T21:55:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6404ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64053e0 0x555ac6405cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac6406750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac635c0f0 con 0x555ac64062d0 2024-05-08T21:55:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac6359ab0 con 0x555ac64053e0 2024-05-08T21:55:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac63553a0 con 0x555ac6404a80 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.135+0000 7f1c08caa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6404ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6404ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:58290/0 (socket says 172.21.15.64:58290) 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 -- 172.21.15.64:0/3221957683 learned_addr learned my addr 172.21.15.64:0/3221957683 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac6406750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c094ab700 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64053e0 0x555ac6405cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 -- 172.21.15.64:0/3221957683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64053e0 msgr2=0x555ac6405cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64053e0 0x555ac6405cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 -- 172.21.15.64:0/3221957683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 msgr2=0x555ac6406750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac6406750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1c08caa700 1 -- 172.21.15.64:0/3221957683 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ac61d6350 con 0x555ac6404a80 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.136+0000 7f1bfa7fc700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1bf00a11a0 0x7f1bf00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.137+0000 7f1bfa7fc700 1 -- 172.21.15.64:0/1049125721 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bfc0cbcc0 con 0x7f1c041037f0 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.137+0000 7f1c08caa700 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6404ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4001dc0 tx=0x7f1bf4009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886aaa4383f8197c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.137+0000 7f1c094ab700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1bf00a11a0 0x7f1bf00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.138+0000 7f1bef7fe700 1 -- 172.21.15.64:0/3221957683 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4015d80 con 0x555ac6404a80 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.138+0000 7f1bef7fe700 1 -- 172.21.15.64:0/3221957683 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bf4007120 con 0x555ac6404a80 2024-05-08T21:55:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.138+0000 7f1bef7fe700 1 -- 172.21.15.64:0/3221957683 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4004ac0 con 0x555ac6404a80 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.138+0000 7f1c094ab700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1bf00a11a0 0x7f1bf00a3660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098e0 tx=0x7f1c00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.138+0000 7f1c178c9b80 1 -- 172.21.15.64:0/3221957683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 msgr2=0x555ac6404ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.138+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6404ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4001dc0 tx=0x7f1bf4009ef0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 -- 172.21.15.64:0/3221957683 shutdown_connections 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac6406750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6404ea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/3221957683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64053e0 0x555ac6405cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 -- 172.21.15.64:0/3221957683 >> 172.21.15.64:0/3221957683 conn(0x555ac63631b0 msgr2=0x555ac6401440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 -- 172.21.15.64:0/3221957683 shutdown_connections 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 -- 172.21.15.64:0/3221957683 wait complete. 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 -- start start 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.139+0000 7f1c178c9b80 1 Processor -- start 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac6404a80 0x555ac640e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac640eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac6415f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac62f1500 con 0x555ac6404a80 2024-05-08T21:55:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac62f11a0 con 0x555ac64062d0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac61d8640 con 0x555ac64053e0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c178c9b80 10 client.0 _ll_register_callbacks cb 0x555ac62faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac6415f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac6415f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40178/0 (socket says 172.21.15.64:40178) 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 -- 172.21.15.64:0/2302730786 learned_addr learned my addr 172.21.15.64:0/2302730786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c08caa700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac640eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac6404a80 0x555ac640e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 -- 172.21.15.64:0/2302730786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 msgr2=0x555ac640eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac640eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 -- 172.21.15.64:0/2302730786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac6404a80 msgr2=0x555ac640e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac6404a80 0x555ac640e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 -- 172.21.15.64:0/2302730786 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf4015270 con 0x555ac64062d0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac6415f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009c10 tx=0x7f1bfc098d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea05e062a7c58b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.140+0000 7f1becff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1becff9700 1 -- 172.21.15.64:0/2302730786 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc09b070 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1becff9700 1 -- 172.21.15.64:0/2302730786 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bfc004ad0 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ac62ae210 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1becff9700 1 -- 172.21.15.64:0/2302730786 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bfc01a2d0 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 10 client.25124 fetch_fsmap learned FSMap version 33 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555ac6182b00 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1becff9700 1 -- 172.21.15.64:0/2302730786 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ac62ae210 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1becff9700 1 -- 172.21.15.64:0/2302730786 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f1bfc0986a0 con 0x555ac64062d0 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 10 client.25124 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 10 client.25124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 20 client.25124 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac641a7e0 0x555ac641cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 4 client.25124 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:55:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.141+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ac63d47b0 con 0x555ac641a7e0 2024-05-08T21:55:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.142+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac641a7e0 0x555ac641cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.145+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac641a7e0 0x555ac641cc00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0931c1871e9bcf5 server_cookie=e0d39b01c5804baf in_seq=0 out_seq=0 2024-05-08T21:55:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.145+0000 7f1becff9700 10 client.25124 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1becff9700 1 -- 172.21.15.64:0/2302730786 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ac63d47b0 con 0x555ac641a7e0 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1becff9700 10 client.25124 handle_command_reply: tid=0 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 client.25124 shutdown 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 msgr2=0x555ac6415f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac6415f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009c10 tx=0x7f1bfc098d50 comp rx=0 tx=0).stop 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 shutdown_connections 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac6415f00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac641a7e0 0x555ac641cc00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac6404a80 0x555ac640e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2302730786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac640eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 >> 172.21.15.64:0/2302730786 conn(0x555ac63631b0 msgr2=0x555ac63ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1becff9700 0 client.25124 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 shutdown_connections 2024-05-08T21:55:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.146+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2302730786 wait complete. 2024-05-08T21:55:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.147+0000 7f1c178c9b80 20 client.25124 trim_cache size 0 max 16384 2024-05-08T21:55:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 Processor -- start 2024-05-08T21:55:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 -- start start 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac6404a80 0x555ac64066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac6415ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac641c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac640d3c0 con 0x555ac64062d0 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac64170b0 con 0x555ac6404a80 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.167+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac6417c10 con 0x555ac64053e0 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac641c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac641c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34134/0 (socket says 172.21.15.64:34134) 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 -- 172.21.15.64:0/2469331911 learned_addr learned my addr 172.21.15.64:0/2469331911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c08caa700 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac6415ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac6404a80 0x555ac64066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 -- 172.21.15.64:0/2469331911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 msgr2=0x555ac6415ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac6415ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 -- 172.21.15.64:0/2469331911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac6404a80 msgr2=0x555ac64066b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac6404a80 0x555ac64066b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 -- 172.21.15.64:0/2469331911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ac631b500 con 0x555ac64062d0 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac6404a80 0x555ac64066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac641c7f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0c9820 tx=0x7f1bfc0976d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96868a705bf873d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1beffff700 1 -- 172.21.15.64:0/2469331911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc09b030 con 0x555ac64062d0 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1beffff700 1 -- 172.21.15.64:0/2469331911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc004ad0 con 0x555ac64062d0 2024-05-08T21:55:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.168+0000 7f1beffff700 1 -- 172.21.15.64:0/2469331911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc098850 con 0x555ac64062d0 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2469331911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 msgr2=0x555ac641c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac641c7f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0c9820 tx=0x7f1bfc0976d0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2469331911 shutdown_connections 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac6404a80 0x555ac64066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64062d0 0x555ac641c7f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2469331911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac64053e0 0x555ac6415ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2469331911 >> 172.21.15.64:0/2469331911 conn(0x555ac63631b0 msgr2=0x555ac640c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.169+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2469331911 shutdown_connections 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2469331911 wait complete. 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 -- start start 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 Processor -- start 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6411130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64053e0 0x555ac6411670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac64145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac6414af0 con 0x555ac64053e0 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac6414c60 con 0x555ac64062d0 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.170+0000 7f1c178c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac6414e20 con 0x555ac6404a80 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c178c9b80 10 client.0 _ll_register_callbacks cb 0x555ac62faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c08caa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64053e0 0x555ac6411670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c08caa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64053e0 0x555ac6411670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34136/0 (socket says 172.21.15.64:34136) 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac64145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c08caa700 1 -- 172.21.15.64:0/2041862134 learned_addr learned my addr 172.21.15.64:0/2041862134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 -- 172.21.15.64:0/2041862134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 msgr2=0x555ac6411130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6411130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 -- 172.21.15.64:0/2041862134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64053e0 msgr2=0x555ac6411670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64053e0 0x555ac6411670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 -- 172.21.15.64:0/2041862134 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ac6182b00 con 0x555ac64062d0 2024-05-08T21:55:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1c09cac700 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac64145b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc095b90 tx=0x7f1bfc0a13a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5bddd44b14fc5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1becff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:55:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.171+0000 7f1becff9700 1 -- 172.21.15.64:0/2041862134 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc09b070 con 0x555ac64062d0 2024-05-08T21:55:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.172+0000 7f1becff9700 1 -- 172.21.15.64:0/2041862134 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bfc09c070 con 0x555ac64062d0 2024-05-08T21:55:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.172+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bf4007120 con 0x555ac64062d0 2024-05-08T21:55:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.172+0000 7f1becff9700 1 -- 172.21.15.64:0/2041862134 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bfc098c60 con 0x555ac64062d0 2024-05-08T21:55:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.172+0000 7f1c178c9b80 10 client.25127 fetch_fsmap learned FSMap version 33 2024-05-08T21:55:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.172+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555ac61dac70 con 0x555ac64062d0 2024-05-08T21:55:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.172+0000 7f1becff9700 1 -- 172.21.15.64:0/2041862134 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc0a5070 con 0x555ac64062d0 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1becff9700 1 -- 172.21.15.64:0/2041862134 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f1bfc0980f0 con 0x555ac64062d0 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1c178c9b80 10 client.25127 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1c178c9b80 10 client.25127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1c178c9b80 20 client.25127 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac6481ce0 0x555ac64840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1c178c9b80 4 client.25127 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.175+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ac634de60 con 0x555ac6481ce0 2024-05-08T21:55:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.176+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac6481ce0 0x555ac64840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.177+0000 7f1becff9700 10 client.25127 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.177+0000 7f1c094ab700 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac6481ce0 0x555ac64840d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd3fc091940177f server_cookie=ef6578379fdcee3d in_seq=0 out_seq=0 2024-05-08T21:55:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1becff9700 1 -- 172.21.15.64:0/2041862134 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ac634de60 con 0x555ac6481ce0 2024-05-08T21:55:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1becff9700 10 client.25127 handle_command_reply: tid=0 2024-05-08T21:55:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 client.25127 shutdown 2024-05-08T21:55:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 msgr2=0x555ac64145b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac64145b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc095b90 tx=0x7f1bfc0a13a0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 shutdown_connections 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555ac64062d0 0x555ac64145b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x555ac6481ce0 0x555ac64840d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555ac6404a80 0x555ac6411130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 --2- 172.21.15.64:0/2041862134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac64053e0 0x555ac6411670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 >> 172.21.15.64:0/2041862134 conn(0x555ac63631b0 msgr2=0x555ac641c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1becff9700 0 client.25127 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.216+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 shutdown_connections 2024-05-08T21:55:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.217+0000 7f1c178c9b80 1 -- 172.21.15.64:0/2041862134 wait complete. 2024-05-08T21:55:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.217+0000 7f1c178c9b80 20 client.25127 trim_cache size 0 max 16384 2024-05-08T21:55:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:55:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:55:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:55:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.217+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1bf00a11a0 msgr2=0x7f1bf00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.217+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1bf00a11a0 0x7f1bf00a3660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098e0 tx=0x7f1c00005fd0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.217+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 msgr2=0x7f1c04103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.217+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 0x7f1c04103c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00ef30 tx=0x7f1bfc00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.218+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 shutdown_connections 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.218+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1bf00a11a0 0x7f1bf00a3660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.218+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c041037f0 0x7f1c04103c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.218+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c041021a0 0x7f1c04106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.218+0000 7f1c0b70f700 1 --2- 172.21.15.64:0/1049125721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c04102b40 0x7f1c04107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.218+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 >> 172.21.15.64:0/1049125721 conn(0x7f1c040fab20 msgr2=0x7f1c04107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.219+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 shutdown_connections 2024-05-08T21:55:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:44.219+0000 7f1c0b70f700 1 -- 172.21.15.64:0/1049125721 wait complete. 2024-05-08T21:55:44.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:44 smithi160 ceph-mon[46962]: pgmap v292: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 54 MiB/s wr, 2.23k op/s 2024-05-08T21:55:44.394 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:44 smithi186 ceph-mon[46619]: pgmap v292: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 54 MiB/s wr, 2.23k op/s 2024-05-08T21:55:44.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:44 smithi064 ceph-mon[42500]: pgmap v292: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 54 MiB/s wr, 2.23k op/s 2024-05-08T21:55:46.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:46 smithi160 ceph-mon[46962]: pgmap v293: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 55 MiB/s wr, 2.27k op/s 2024-05-08T21:55:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:46 smithi186 ceph-mon[46619]: pgmap v293: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 55 MiB/s wr, 2.27k op/s 2024-05-08T21:55:46.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:46 smithi064 ceph-mon[42500]: pgmap v293: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 55 MiB/s wr, 2.27k op/s 2024-05-08T21:55:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 55.0 s, 4528.2 tps, lat 22.067 ms stddev 24.822 2024-05-08T21:55:48.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:48 smithi160 ceph-mon[46962]: pgmap v294: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 50 MiB/s wr, 2.07k op/s 2024-05-08T21:55:48.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:48 smithi186 ceph-mon[46619]: pgmap v294: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 50 MiB/s wr, 2.07k op/s 2024-05-08T21:55:48.470 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:48 smithi064 ceph-mon[42500]: pgmap v294: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 50 MiB/s wr, 2.07k op/s 2024-05-08T21:55:50.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:50 smithi160 ceph-mon[46962]: pgmap v295: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 45 MiB/s wr, 1.96k op/s 2024-05-08T21:55:50.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:50 smithi186 ceph-mon[46619]: pgmap v295: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 45 MiB/s wr, 1.96k op/s 2024-05-08T21:55:50.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:50 smithi064 ceph-mon[42500]: pgmap v295: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 45 MiB/s wr, 1.96k op/s 2024-05-08T21:55:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 60.0 s, 4014.2 tps, lat 24.844 ms stddev 26.226 2024-05-08T21:55:52.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:52 smithi160 ceph-mon[46962]: pgmap v296: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-05-08T21:55:52.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:52 smithi186 ceph-mon[46619]: pgmap v296: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-05-08T21:55:52.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:52 smithi064 ceph-mon[42500]: pgmap v296: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-05-08T21:55:54.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:54 smithi160 ceph-mon[46962]: pgmap v297: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 25 MiB/s wr, 1.05k op/s 2024-05-08T21:55:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:54 smithi186 ceph-mon[46619]: pgmap v297: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 25 MiB/s wr, 1.05k op/s 2024-05-08T21:55:54.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:54 smithi064 ceph-mon[42500]: pgmap v297: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 25 MiB/s wr, 1.05k op/s 2024-05-08T21:55:56.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:56 smithi160 ceph-mon[46962]: pgmap v298: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 30 MiB/s wr, 1.21k op/s 2024-05-08T21:55:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:56 smithi186 ceph-mon[46619]: pgmap v298: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 30 MiB/s wr, 1.21k op/s 2024-05-08T21:55:56.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:56 smithi064 ceph-mon[42500]: pgmap v298: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 30 MiB/s wr, 1.21k op/s 2024-05-08T21:55:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 65.0 s, 3763.0 tps, lat 26.602 ms stddev 29.444 2024-05-08T21:55:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.491+0000 7efe95704700 1 -- 172.21.15.64:0/3400603000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 msgr2=0x7efe90100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.491+0000 7efe95704700 1 --2- 172.21.15.64:0/3400603000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90100240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efe84009990 tx=0x7efe84009ca0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.515+0000 7efe95704700 1 -- 172.21.15.64:0/3400603000 shutdown_connections 2024-05-08T21:55:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.516+0000 7efe95704700 1 --2- 172.21.15.64:0/3400603000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe90101930 0x7efe90108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.516+0000 7efe95704700 1 --2- 172.21.15.64:0/3400603000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90100240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.516+0000 7efe95704700 1 --2- 172.21.15.64:0/3400603000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90100f70 0x7efe901013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.516+0000 7efe95704700 1 -- 172.21.15.64:0/3400603000 >> 172.21.15.64:0/3400603000 conn(0x7efe900fb390 msgr2=0x7efe900fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.526+0000 7efe95704700 1 -- 172.21.15.64:0/3400603000 shutdown_connections 2024-05-08T21:55:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.529+0000 7efe95704700 1 -- 172.21.15.64:0/3400603000 wait complete. 2024-05-08T21:55:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.530+0000 7efe95704700 1 Processor -- start 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.545+0000 7efe95704700 1 -- start start 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.546+0000 7efe95704700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.546+0000 7efe95704700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe90100f70 0x7efe90070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.546+0000 7efe95704700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90101930 0x7efe90071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.546+0000 7efe95704700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900725a0 con 0x7efe900ffe20 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.546+0000 7efe95704700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900726e0 con 0x7efe90101930 2024-05-08T21:55:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.546+0000 7efe95704700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90072870 con 0x7efe90100f70 2024-05-08T21:55:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.551+0000 7efe8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.551+0000 7efe8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38128/0 (socket says 172.21.15.64:38128) 2024-05-08T21:55:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.551+0000 7efe8effd700 1 -- 172.21.15.64:0/441481013 learned_addr learned my addr 172.21.15.64:0/441481013 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.551+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe90100f70 0x7efe90070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.553+0000 7efe8f7fe700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90101930 0x7efe90071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.554+0000 7efe8f7fe700 1 -- 172.21.15.64:0/441481013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe90100f70 msgr2=0x7efe90070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.554+0000 7efe8f7fe700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe90100f70 0x7efe90070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.554+0000 7efe8f7fe700 1 -- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 msgr2=0x7efe90070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.554+0000 7efe8f7fe700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.554+0000 7efe8f7fe700 1 -- 172.21.15.64:0/441481013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe840095f0 con 0x7efe90101930 2024-05-08T21:55:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.555+0000 7efe8f7fe700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90101930 0x7efe90071fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe88009a50 tx=0x7efe88009e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.562+0000 7efe77fff700 1 -- 172.21.15.64:0/441481013 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880074b0 con 0x7efe90101930 2024-05-08T21:55:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.562+0000 7efe77fff700 1 -- 172.21.15.64:0/441481013 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8800b0b0 con 0x7efe90101930 2024-05-08T21:55:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.562+0000 7efe77fff700 1 -- 172.21.15.64:0/441481013 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800d6f0 con 0x7efe90101930 2024-05-08T21:55:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.564+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe90072b50 con 0x7efe90101930 2024-05-08T21:55:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.564+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe90072fe0 con 0x7efe90101930 2024-05-08T21:55:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.576+0000 7efea18beb80 1 Processor -- start 2024-05-08T21:55:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 -- start start 2024-05-08T21:55:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x564588596ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x564588597cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x564588598750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645884ee0f0 con 0x564588596a80 2024-05-08T21:55:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645884ebab0 con 0x5645885973e0 2024-05-08T21:55:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efea18beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645884e73a0 con 0x5645885982d0 2024-05-08T21:55:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.577+0000 7efe77fff700 1 -- 172.21.15.64:0/441481013 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efe88014030 con 0x7efe90101930 2024-05-08T21:55:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.579+0000 7efe8effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x564588597cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.579+0000 7efe8effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x564588597cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:49450/0 (socket says 172.21.15.64:49450) 2024-05-08T21:55:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.579+0000 7efe8effd700 1 -- 172.21.15.64:0/1865078653 learned_addr learned my addr 172.21.15.64:0/1865078653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.580+0000 7efe8effd700 1 -- 172.21.15.64:0/1865078653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 msgr2=0x564588598750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.580+0000 7efe8effd700 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x564588598750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.580+0000 7efe8effd700 1 -- 172.21.15.64:0/1865078653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 msgr2=0x564588596ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.580+0000 7efe8effd700 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x564588596ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.580+0000 7efe8effd700 1 -- 172.21.15.64:0/1865078653 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564588368350 con 0x5645885973e0 2024-05-08T21:55:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.582+0000 7efe8effd700 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x564588597cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe8400d760 tx=0x7efe8400d790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b512000f0d3e7842 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.582+0000 7efe757fa700 1 -- 172.21.15.64:0/1865078653 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8400d230 con 0x5645885973e0 2024-05-08T21:55:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.583+0000 7efe757fa700 1 -- 172.21.15.64:0/1865078653 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8401d040 con 0x5645885973e0 2024-05-08T21:55:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.583+0000 7efe757fa700 1 -- 172.21.15.64:0/1865078653 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84004e40 con 0x5645885973e0 2024-05-08T21:55:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.590+0000 7efea18beb80 1 -- 172.21.15.64:0/1865078653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 msgr2=0x564588597cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.590+0000 7efea18beb80 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x564588597cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe8400d760 tx=0x7efe8400d790 comp rx=0 tx=0).stop 2024-05-08T21:55:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.590+0000 7efe77fff700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe780a1320 0x7efe780a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.590+0000 7efe77fff700 1 -- 172.21.15.64:0/441481013 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe880cc400 con 0x7efe90101930 2024-05-08T21:55:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.590+0000 7efea18beb80 1 -- 172.21.15.64:0/1865078653 shutdown_connections 2024-05-08T21:55:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.590+0000 7efea18beb80 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x564588598750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.591+0000 7efea18beb80 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x564588596ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.591+0000 7efea18beb80 1 --2- 172.21.15.64:0/1865078653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x564588597cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.591+0000 7efea18beb80 1 -- 172.21.15.64:0/1865078653 >> 172.21.15.64:0/1865078653 conn(0x5645884f51b0 msgr2=0x564588593440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.599+0000 7efea18beb80 1 -- 172.21.15.64:0/1865078653 shutdown_connections 2024-05-08T21:55:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.599+0000 7efea18beb80 1 -- 172.21.15.64:0/1865078653 wait complete. 2024-05-08T21:55:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.599+0000 7efea18beb80 1 -- start start 2024-05-08T21:55:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 Processor -- start 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x5645885a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x5645885a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564588483500 con 0x564588596a80 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645884831a0 con 0x5645885973e0 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458836a640 con 0x5645885982d0 2024-05-08T21:55:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.600+0000 7efea18beb80 10 client.0 _ll_register_callbacks cb 0x56458848cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.602+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe780a1320 0x7efe780a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.604+0000 7efe8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x5645885a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.604+0000 7efe8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x5645885a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38144/0 (socket says 172.21.15.64:38144) 2024-05-08T21:55:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.604+0000 7efe8effd700 1 -- 172.21.15.64:0/2164519488 learned_addr learned my addr 172.21.15.64:0/2164519488 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.604+0000 7efe8f7fe700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x5645885a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.606+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe8effd700 1 -- 172.21.15.64:0/2164519488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 msgr2=0x5645885a7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe8effd700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x5645885a7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe8effd700 1 -- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 msgr2=0x5645885a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe8effd700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe8effd700 1 -- 172.21.15.64:0/2164519488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe90108b80 con 0x564588596a80 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe8effd700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x5645885a08a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe84006010 tx=0x7efe84023b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637df31e35d20d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.607+0000 7efe72ff5700 1 -- 172.21.15.64:0/2164519488 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84015850 con 0x564588596a80 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.608+0000 7efe72ff5700 1 -- 172.21.15.64:0/2164519488 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe840159b0 con 0x564588596a80 2024-05-08T21:55:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.608+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564588440210 con 0x564588596a80 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.608+0000 7efe72ff5700 1 -- 172.21.15.64:0/2164519488 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe840047e0 con 0x564588596a80 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.608+0000 7efea18beb80 10 client.15414 fetch_fsmap learned FSMap version 33 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.608+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564588314b00 con 0x564588596a80 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efe72ff5700 1 -- 172.21.15.64:0/2164519488 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564588440210 con 0x564588596a80 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efe72ff5700 1 -- 172.21.15.64:0/2164519488 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7efe84032070 con 0x564588596a80 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efea18beb80 10 client.15414 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efea18beb80 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efea18beb80 20 client.15414 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efea18beb80 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645885ac3c0 0x5645885ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efea18beb80 4 client.15414 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:55:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.609+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645885667b0 con 0x5645885ac3c0 2024-05-08T21:55:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.610+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645885ac3c0 0x5645885ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.617+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe780a1320 0x7efe780a37e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efe80006ae0 tx=0x7efe800092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.618+0000 7efe72ff5700 10 client.15414 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.618+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645885ac3c0 0x5645885ae840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e67ec769811135 server_cookie=624a88390d42388b in_seq=0 out_seq=0 2024-05-08T21:55:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.621+0000 7efe72ff5700 1 -- 172.21.15.64:0/2164519488 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645885667b0 con 0x5645885ac3c0 2024-05-08T21:55:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.621+0000 7efe72ff5700 10 client.15414 handle_command_reply: tid=0 2024-05-08T21:55:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.623+0000 7efea18beb80 1 client.15414 shutdown 2024-05-08T21:55:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.623+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 msgr2=0x5645885a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.623+0000 7efea18beb80 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x5645885a08a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe84006010 tx=0x7efe84023b40 comp rx=0 tx=0).stop 2024-05-08T21:55:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 shutdown_connections 2024-05-08T21:55:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efea18beb80 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645885ac3c0 0x5645885ae840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efea18beb80 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645885982d0 0x5645885a7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efea18beb80 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564588596a80 0x5645885a08a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efea18beb80 1 --2- 172.21.15.64:0/2164519488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efe72ff5700 0 client.15414 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.629+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 >> 172.21.15.64:0/2164519488 conn(0x5645884f51b0 msgr2=0x564588591860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.630+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 shutdown_connections 2024-05-08T21:55:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.630+0000 7efea18beb80 1 -- 172.21.15.64:0/2164519488 wait complete. 2024-05-08T21:55:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.641+0000 7efea18beb80 20 client.15414 trim_cache size 0 max 16384 2024-05-08T21:55:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 Processor -- start 2024-05-08T21:55:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 -- start start 2024-05-08T21:55:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x564588596e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885982d0 0x5645885a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645885a1a90 con 0x5645885982d0 2024-05-08T21:55:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458859e4f0 con 0x5645885973e0 2024-05-08T21:55:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efea18beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645885a8c00 con 0x564588596a80 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efe8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885982d0 0x5645885a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efe8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885982d0 0x5645885a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38152/0 (socket says 172.21.15.64:38152) 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efe8f7fe700 1 -- 172.21.15.64:0/1848666442 learned_addr learned my addr 172.21.15.64:0/1848666442 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efe8effd700 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x564588596e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.668+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.669+0000 7efe8effd700 1 -- 172.21.15.64:0/1848666442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 msgr2=0x5645885ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.669+0000 7efe8effd700 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.669+0000 7efe8effd700 1 -- 172.21.15.64:0/1848666442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885982d0 msgr2=0x5645885a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.669+0000 7efe8effd700 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885982d0 0x5645885a7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.669+0000 7efe8effd700 1 -- 172.21.15.64:0/1848666442 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645884ad500 con 0x564588596a80 2024-05-08T21:55:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.670+0000 7efe8effd700 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x564588596e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe8400dd70 tx=0x7efe84029ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a3ae00235a664c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.671+0000 7efe75ffb700 1 -- 172.21.15.64:0/1848666442 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840049b0 con 0x564588596a80 2024-05-08T21:55:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.671+0000 7efe75ffb700 1 -- 172.21.15.64:0/1848666442 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe84019040 con 0x564588596a80 2024-05-08T21:55:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.671+0000 7efe75ffb700 1 -- 172.21.15.64:0/1848666442 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840159d0 con 0x564588596a80 2024-05-08T21:55:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.672+0000 7efea18beb80 1 -- 172.21.15.64:0/1848666442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 msgr2=0x564588596e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.672+0000 7efea18beb80 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x564588596e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe8400dd70 tx=0x7efe84029ba0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 -- 172.21.15.64:0/1848666442 shutdown_connections 2024-05-08T21:55:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885982d0 0x5645885a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x564588596e60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 --2- 172.21.15.64:0/1848666442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885973e0 0x5645885ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 -- 172.21.15.64:0/1848666442 >> 172.21.15.64:0/1848666442 conn(0x5645884f51b0 msgr2=0x56458859fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 -- 172.21.15.64:0/1848666442 shutdown_connections 2024-05-08T21:55:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.679+0000 7efea18beb80 1 -- 172.21.15.64:0/1848666442 wait complete. 2024-05-08T21:55:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 -- start start 2024-05-08T21:55:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 Processor -- start 2024-05-08T21:55:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x5645885a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 0x5645885a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885982d0 0x5645885a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564588600940 con 0x5645885973e0 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564588600a80 con 0x5645885982d0 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.680+0000 7efea18beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564588600c40 con 0x564588596a80 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.681+0000 7efea18beb80 10 client.0 _ll_register_callbacks cb 0x56458848cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.682+0000 7efe8e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 0x5645885a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.682+0000 7efe8e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 0x5645885a4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38164/0 (socket says 172.21.15.64:38164) 2024-05-08T21:55:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.682+0000 7efe8e7fc700 1 -- 172.21.15.64:0/487257954 learned_addr learned my addr 172.21.15.64:0/487257954 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:55:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.687+0000 7efe8f7fe700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885982d0 0x5645885a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.688+0000 7efe8effd700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x5645885a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.689+0000 7efe8e7fc700 1 -- 172.21.15.64:0/487257954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 msgr2=0x5645885a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.689+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x5645885a7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.689+0000 7efe8e7fc700 1 -- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885982d0 msgr2=0x5645885a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:55:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.689+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885982d0 0x5645885a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.689+0000 7efe8e7fc700 1 -- 172.21.15.64:0/487257954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564588314b00 con 0x5645885973e0 2024-05-08T21:55:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.695+0000 7efe8e7fc700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 0x5645885a4fc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efe80006e50 tx=0x7efe80006a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dccb16cbbfe537 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:55:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.695+0000 7efe72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:55:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.699+0000 7efe72ff5700 1 -- 172.21.15.64:0/487257954 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80018530 con 0x5645885973e0 2024-05-08T21:55:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.699+0000 7efe72ff5700 1 -- 172.21.15.64:0/487257954 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe80018690 con 0x5645885973e0 2024-05-08T21:55:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.699+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe8401d040 con 0x5645885973e0 2024-05-08T21:55:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.700+0000 7efe72ff5700 1 -- 172.21.15.64:0/487257954 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe80018850 con 0x5645885973e0 2024-05-08T21:55:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 10 client.15420 fetch_fsmap learned FSMap version 33 2024-05-08T21:55:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56458836cc70 con 0x5645885973e0 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efe72ff5700 1 -- 172.21.15.64:0/487257954 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8401d040 con 0x5645885973e0 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efe72ff5700 1 -- 172.21.15.64:0/487257954 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7efe80037070 con 0x5645885973e0 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 10 client.15420 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 20 client.15420 populate_metadata read hostname 'smithi064' 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645886246f0 0x564588626b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 4 client.15420 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:55:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.701+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645884dfe60 con 0x5645886246f0 2024-05-08T21:55:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.705+0000 7efe8effd700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645886246f0 0x564588626b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:55:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.707+0000 7efe8effd700 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645886246f0 0x564588626b10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea5c5760ff8d3f9 server_cookie=6182e5e32b7b48e2 in_seq=0 out_seq=0 2024-05-08T21:55:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.707+0000 7efe72ff5700 10 client.15420 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.747+0000 7efe72ff5700 1 -- 172.21.15.64:0/487257954 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645884dfe60 con 0x5645886246f0 2024-05-08T21:55:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.747+0000 7efe72ff5700 10 client.15420 handle_command_reply: tid=0 2024-05-08T21:55:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.747+0000 7efea18beb80 1 client.15420 shutdown 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 msgr2=0x5645885a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 0x5645885a4fc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efe80006e50 tx=0x7efe80006a70 comp rx=0 tx=0).stop 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 shutdown_connections 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5645885982d0 0x5645885a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5645886246f0 0x564588626b10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564588596a80 0x5645885a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 --2- 172.21.15.64:0/487257954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645885973e0 0x5645885a4fc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 >> 172.21.15.64:0/487257954 conn(0x5645884f51b0 msgr2=0x56458859e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efe72ff5700 0 client.15420 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 shutdown_connections 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 1 -- 172.21.15.64:0/487257954 wait complete. 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.748+0000 7efea18beb80 20 client.15420 trim_cache size 0 max 16384 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:55:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe780a1320 msgr2=0x7efe780a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe780a1320 0x7efe780a37e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efe80006ae0 tx=0x7efe800092c0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90101930 msgr2=0x7efe90071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90101930 0x7efe90071fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe88009a50 tx=0x7efe88009e10 comp rx=0 tx=0).stop 2024-05-08T21:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 shutdown_connections 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efe780a1320 0x7efe780a37e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe90101930 0x7efe90071fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe900ffe20 0x7efe90070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 --2- 172.21.15.64:0/441481013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe90100f70 0x7efe90070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 >> 172.21.15.64:0/441481013 conn(0x7efe900fb390 msgr2=0x7efe90104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 shutdown_connections 2024-05-08T21:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:55:57.749+0000 7efe95704700 1 -- 172.21.15.64:0/441481013 wait complete. 2024-05-08T21:55:58.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:58 smithi186 ceph-mon[46619]: pgmap v299: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-05-08T21:55:58.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:58 smithi064 ceph-mon[42500]: pgmap v299: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-05-08T21:55:58.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:58 smithi160 ceph-mon[46962]: pgmap v299: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-05-08T21:55:59.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:55:59 smithi186 ceph-mon[46619]: pgmap v300: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-05-08T21:55:59.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:55:59 smithi064 ceph-mon[42500]: pgmap v300: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-05-08T21:55:59.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:55:59 smithi160 ceph-mon[46962]: pgmap v300: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-05-08T21:56:01.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:01 smithi064 ceph-mon[42500]: pgmap v301: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 50 MiB/s wr, 2.04k op/s 2024-05-08T21:56:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 70.0 s, 3311.2 tps, lat 30.118 ms stddev 53.059 2024-05-08T21:56:01.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:01 smithi160 ceph-mon[46962]: pgmap v301: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 50 MiB/s wr, 2.04k op/s 2024-05-08T21:56:01.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:01 smithi186 ceph-mon[46619]: pgmap v301: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 50 MiB/s wr, 2.04k op/s 2024-05-08T21:56:03.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:03 smithi064 ceph-mon[42500]: pgmap v302: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 55 MiB/s wr, 2.24k op/s 2024-05-08T21:56:03.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:03 smithi160 ceph-mon[46962]: pgmap v302: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 55 MiB/s wr, 2.24k op/s 2024-05-08T21:56:03.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:03 smithi186 ceph-mon[46619]: pgmap v302: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 55 MiB/s wr, 2.24k op/s 2024-05-08T21:56:05.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:05 smithi064 ceph-mon[42500]: pgmap v303: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 56 MiB/s wr, 2.27k op/s 2024-05-08T21:56:05.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:05 smithi160 ceph-mon[46962]: pgmap v303: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 56 MiB/s wr, 2.27k op/s 2024-05-08T21:56:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:05 smithi186 ceph-mon[46619]: pgmap v303: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 56 MiB/s wr, 2.27k op/s 2024-05-08T21:56:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 75.0 s, 4597.6 tps, lat 21.872 ms stddev 25.604 2024-05-08T21:56:07.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:07 smithi160 ceph-mon[46962]: pgmap v304: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 49 MiB/s wr, 2.05k op/s 2024-05-08T21:56:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:07 smithi186 ceph-mon[46619]: pgmap v304: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 49 MiB/s wr, 2.05k op/s 2024-05-08T21:56:07.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:07 smithi064 ceph-mon[42500]: pgmap v304: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 49 MiB/s wr, 2.05k op/s 2024-05-08T21:56:09.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:09 smithi160 ceph-mon[46962]: pgmap v305: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 47 MiB/s wr, 2.02k op/s 2024-05-08T21:56:09.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:09 smithi186 ceph-mon[46619]: pgmap v305: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 47 MiB/s wr, 2.02k op/s 2024-05-08T21:56:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:09 smithi064 ceph-mon[42500]: pgmap v305: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 47 MiB/s wr, 2.02k op/s 2024-05-08T21:56:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.918+0000 7fcf4435d700 1 -- 172.21.15.64:0/3321581424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1009d0 msgr2=0x7fcf3c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.918+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3321581424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c108270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400b0a0 tx=0x7fcf3400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 -- 172.21.15.64:0/3321581424 shutdown_connections 2024-05-08T21:56:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3321581424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c108270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3321581424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1013c0 0x7fcf3c1017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3321581424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c100010 0x7fcf3c100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 -- 172.21.15.64:0/3321581424 >> 172.21.15.64:0/3321581424 conn(0x7fcf3c0fab20 msgr2=0x7fcf3c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 -- 172.21.15.64:0/3321581424 shutdown_connections 2024-05-08T21:56:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.919+0000 7fcf4435d700 1 -- 172.21.15.64:0/3321581424 wait complete. 2024-05-08T21:56:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.920+0000 7fcf4435d700 1 Processor -- start 2024-05-08T21:56:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.920+0000 7fcf4435d700 1 -- start start 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.920+0000 7fcf4435d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c100010 0x7fcf3c10ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf4435d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c10f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf4435d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1013c0 0x7fcf3c110240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf4435d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c110810 con 0x7fcf3c100010 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf4435d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c110980 con 0x7fcf3c1009d0 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf4435d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c110b40 con 0x7fcf3c1013c0 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf418f8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c10f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf418f8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c10f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:39842/0 (socket says 172.21.15.64:39842) 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf418f8700 1 -- 172.21.15.64:0/3746236781 learned_addr learned my addr 172.21.15.64:0/3746236781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1013c0 0x7fcf3c110240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c100010 0x7fcf3c10ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 -- 172.21.15.64:0/3746236781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c1009d0 msgr2=0x7fcf3c10f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c10f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 -- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c100010 msgr2=0x7fcf3c10ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c100010 0x7fcf3c10ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 -- 172.21.15.64:0/3746236781 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34009d00 con 0x7fcf3c1013c0 2024-05-08T21:56:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf428fa700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1013c0 0x7fcf3c110240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf34007c70 tx=0x7fcf340095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.921+0000 7fcf337fe700 1 -- 172.21.15.64:0/3746236781 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400e040 con 0x7fcf3c1013c0 2024-05-08T21:56:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.922+0000 7fcf337fe700 1 -- 172.21.15.64:0/3746236781 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf34003e10 con 0x7fcf3c1013c0 2024-05-08T21:56:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.922+0000 7fcf337fe700 1 -- 172.21.15.64:0/3746236781 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3401da30 con 0x7fcf3c1013c0 2024-05-08T21:56:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.922+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c110d60 con 0x7fcf3c1013c0 2024-05-08T21:56:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.922+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c111220 con 0x7fcf3c1013c0 2024-05-08T21:56:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.924+0000 7fcf337fe700 1 -- 172.21.15.64:0/3746236781 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcf340056c0 con 0x7fcf3c1013c0 2024-05-08T21:56:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.927+0000 7fcf50517b80 1 Processor -- start 2024-05-08T21:56:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.927+0000 7fcf50517b80 1 -- start start 2024-05-08T21:56:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.927+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.927+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2a3e0 0x559cf9e2acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2b2d0 0x559cf9e2b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf50517b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9d810f0 con 0x559cf9e2b2d0 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf50517b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9d7eab0 con 0x559cf9e29a80 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf50517b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9d7a3a0 con 0x559cf9e2a3e0 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf418f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2b2d0 0x559cf9e2b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf418f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2b2d0 0x559cf9e2b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56952/0 (socket says 172.21.15.64:56952) 2024-05-08T21:56:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf418f8700 1 -- 172.21.15.64:0/2711731945 learned_addr learned my addr 172.21.15.64:0/2711731945 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.928+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2a3e0 0x559cf9e2acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf418f8700 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e29ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf337fe700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcf280a1200 0x7fcf280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf420f9700 1 -- 172.21.15.64:0/2711731945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 msgr2=0x559cf9e29ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e29ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf420f9700 1 -- 172.21.15.64:0/2711731945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2b2d0 msgr2=0x559cf9e2b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2b2d0 0x559cf9e2b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf420f9700 1 -- 172.21.15.64:0/2711731945 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf9bfb350 con 0x559cf9e2a3e0 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf337fe700 1 -- 172.21.15.64:0/3746236781 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcf340cdcb0 con 0x7fcf3c1013c0 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.942+0000 7fcf418f8700 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e29ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.943+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2a3e0 0x559cf9e2acf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800bc00 tx=0x7fcf3800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb093da9ab6f129 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.944+0000 7fcf30da4700 1 -- 172.21.15.64:0/2711731945 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38010cd0 con 0x559cf9e2a3e0 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.944+0000 7fcf30da4700 1 -- 172.21.15.64:0/2711731945 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf38015070 con 0x559cf9e2a3e0 2024-05-08T21:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.944+0000 7fcf30da4700 1 -- 172.21.15.64:0/2711731945 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38005020 con 0x559cf9e2a3e0 2024-05-08T21:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.944+0000 7fcf428fa700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcf280a1200 0x7fcf280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 -- 172.21.15.64:0/2711731945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2a3e0 msgr2=0x559cf9e2acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2a3e0 0x559cf9e2acf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800bc00 tx=0x7fcf3800bf10 comp rx=0 tx=0).stop 2024-05-08T21:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf428fa700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcf280a1200 0x7fcf280a36c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf34007b40 tx=0x7fcf3400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 -- 172.21.15.64:0/2711731945 shutdown_connections 2024-05-08T21:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e29ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2b2d0 0x559cf9e2b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2711731945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2a3e0 0x559cf9e2acf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.945+0000 7fcf50517b80 1 -- 172.21.15.64:0/2711731945 >> 172.21.15.64:0/2711731945 conn(0x559cf9d881b0 msgr2=0x559cf9e26440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.949+0000 7fcf50517b80 1 -- 172.21.15.64:0/2711731945 shutdown_connections 2024-05-08T21:56:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.950+0000 7fcf50517b80 1 -- 172.21.15.64:0/2711731945 wait complete. 2024-05-08T21:56:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.950+0000 7fcf50517b80 1 -- start start 2024-05-08T21:56:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 Processor -- start 2024-05-08T21:56:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e33930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 0x559cf9e33e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2b2d0 0x559cf9e3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9d16500 con 0x559cf9e2a3e0 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9d161a0 con 0x559cf9e29a80 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9bfd640 con 0x559cf9e2b2d0 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf50517b80 10 client.0 _ll_register_callbacks cb 0x559cf9d1fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 0x559cf9e33e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 0x559cf9e33e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56956/0 (socket says 172.21.15.64:56956) 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 -- 172.21.15.64:0/3643486426 learned_addr learned my addr 172.21.15.64:0/3643486426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 -- 172.21.15.64:0/3643486426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2b2d0 msgr2=0x559cf9e3aeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2b2d0 0x559cf9e3aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 -- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 msgr2=0x559cf9e33930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e33930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.951+0000 7fcf420f9700 1 -- 172.21.15.64:0/3643486426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3800b860 con 0x559cf9e2a3e0 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.952+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 0x559cf9e33e70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcf38010e70 tx=0x7fcf38010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5e8b8248b8ce16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.952+0000 7fcf25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:56:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.952+0000 7fcf25ffb700 1 -- 172.21.15.64:0/3643486426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38020070 con 0x559cf9e2a3e0 2024-05-08T21:56:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.952+0000 7fcf25ffb700 1 -- 172.21.15.64:0/3643486426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcf38017070 con 0x559cf9e2a3e0 2024-05-08T21:56:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.952+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cf9cd3210 con 0x559cf9e2a3e0 2024-05-08T21:56:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.954+0000 7fcf25ffb700 1 -- 172.21.15.64:0/3643486426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf38004330 con 0x559cf9e2a3e0 2024-05-08T21:56:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.954+0000 7fcf50517b80 10 client.15432 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.954+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559cf9ba7b00 con 0x559cf9e2a3e0 2024-05-08T21:56:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.954+0000 7fcf25ffb700 1 -- 172.21.15.64:0/3643486426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559cf9cd3210 con 0x559cf9e2a3e0 2024-05-08T21:56:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf25ffb700 1 -- 172.21.15.64:0/3643486426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fcf38004d00 con 0x559cf9e2a3e0 2024-05-08T21:56:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf50517b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf50517b80 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf50517b80 20 client.15432 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e3f650 0x559cf9e41a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf50517b80 4 client.15432 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:56:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.955+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cf9df97b0 con 0x559cf9e3f650 2024-05-08T21:56:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.956+0000 7fcf418f8700 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e3f650 0x559cf9e41a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.957+0000 7fcf25ffb700 10 client.15432 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.957+0000 7fcf418f8700 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e3f650 0x559cf9e41a70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdaa55a47d12beca server_cookie=b27aac785dbc143 in_seq=0 out_seq=0 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf25ffb700 1 -- 172.21.15.64:0/3643486426 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559cf9df97b0 con 0x559cf9e3f650 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf25ffb700 10 client.15432 handle_command_reply: tid=0 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 client.15432 shutdown 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 msgr2=0x559cf9e33e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 0x559cf9e33e70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcf38010e70 tx=0x7fcf38010f50 comp rx=0 tx=0).stop 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 shutdown_connections 2024-05-08T21:56:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e33930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e3f650 0x559cf9e41a70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e2b2d0 0x559cf9e3aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3643486426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e2a3e0 0x559cf9e33e70 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcf38010e70 tx=0x7fcf38010f50 comp rx=0 tx=0).stop 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 >> 172.21.15.64:0/3643486426 conn(0x559cf9d881b0 msgr2=0x559cf9e24860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf25ffb700 0 client.15432 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.958+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 shutdown_connections 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.959+0000 7fcf50517b80 1 -- 172.21.15.64:0/3643486426 wait complete. 2024-05-08T21:56:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.959+0000 7fcf50517b80 20 client.15432 trim_cache size 0 max 16384 2024-05-08T21:56:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 Processor -- start 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 -- start start 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e29a80 0x559cf9e3b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 0x559cf9e32e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e3ae70 0x559cf9e2b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9e219b0 con 0x559cf9e3ae70 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9e32170 con 0x559cf9e2b2d0 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf50517b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9e3c030 con 0x559cf9e29a80 2024-05-08T21:56:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf420f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 0x559cf9e32e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf418f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e3ae70 0x559cf9e2b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf420f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 0x559cf9e32e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:39874/0 (socket says 172.21.15.64:39874) 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf420f9700 1 -- 172.21.15.64:0/3883807503 learned_addr learned my addr 172.21.15.64:0/3883807503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.984+0000 7fcf418f8700 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e29a80 0x559cf9e3b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf420f9700 1 -- 172.21.15.64:0/3883807503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e29a80 msgr2=0x559cf9e3b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e29a80 0x559cf9e3b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf420f9700 1 -- 172.21.15.64:0/3883807503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e3ae70 msgr2=0x559cf9e2b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e3ae70 0x559cf9e2b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf420f9700 1 -- 172.21.15.64:0/3883807503 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf9d40500 con 0x559cf9e2b2d0 2024-05-08T21:56:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf418f8700 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e29a80 0x559cf9e3b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:56:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf420f9700 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 0x559cf9e32e40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800bb50 tx=0x7fcf38005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade723eb8c34a425 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf315a5700 1 -- 172.21.15.64:0/3883807503 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38020070 con 0x559cf9e2b2d0 2024-05-08T21:56:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf315a5700 1 -- 172.21.15.64:0/3883807503 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf380047f0 con 0x559cf9e2b2d0 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.985+0000 7fcf315a5700 1 -- 172.21.15.64:0/3883807503 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38004c90 con 0x559cf9e2b2d0 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.986+0000 7fcf50517b80 1 -- 172.21.15.64:0/3883807503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 msgr2=0x559cf9e32e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.986+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 0x559cf9e32e40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800bb50 tx=0x7fcf38005270 comp rx=0 tx=0).stop 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.986+0000 7fcf50517b80 1 -- 172.21.15.64:0/3883807503 shutdown_connections 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.986+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e3ae70 0x559cf9e2b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.986+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e29a80 0x559cf9e3b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.987+0000 7fcf50517b80 1 --2- 172.21.15.64:0/3883807503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e2b2d0 0x559cf9e32e40 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800bb50 tx=0x7fcf38005270 comp rx=0 tx=0).stop 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.987+0000 7fcf50517b80 1 -- 172.21.15.64:0/3883807503 >> 172.21.15.64:0/3883807503 conn(0x559cf9d881b0 msgr2=0x559cf9e318a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.987+0000 7fcf50517b80 1 -- 172.21.15.64:0/3883807503 shutdown_connections 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.987+0000 7fcf50517b80 1 -- 172.21.15.64:0/3883807503 wait complete. 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.987+0000 7fcf50517b80 1 -- start start 2024-05-08T21:56:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.987+0000 7fcf50517b80 1 Processor -- start 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e38050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 0x559cf9e38590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e393b0 0x559cf9e39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9e39d70 con 0x559cf9e393b0 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9e39ee0 con 0x559cf9e29a80 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf9e3a0a0 con 0x559cf9e3ae70 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf50517b80 10 client.0 _ll_register_callbacks cb 0x559cf9d1fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 0x559cf9e38590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 0x559cf9e38590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:57476/0 (socket says 172.21.15.64:57476) 2024-05-08T21:56:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 -- 172.21.15.64:0/2758645608 learned_addr learned my addr 172.21.15.64:0/2758645608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 -- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 msgr2=0x559cf9e38050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e38050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 -- 172.21.15.64:0/2758645608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e393b0 msgr2=0x559cf9e39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e393b0 0x559cf9e39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 -- 172.21.15.64:0/2758645608 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cf9ba7b00 con 0x559cf9e3ae70 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf420f9700 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 0x559cf9e38590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf38005300 tx=0x7fcf38004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0dee60690268395 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.988+0000 7fcf25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.989+0000 7fcf25ffb700 1 -- 172.21.15.64:0/2758645608 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38020040 con 0x559cf9e3ae70 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.989+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf38015070 con 0x559cf9e3ae70 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.989+0000 7fcf25ffb700 1 -- 172.21.15.64:0/2758645608 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcf38017070 con 0x559cf9e3ae70 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.989+0000 7fcf25ffb700 1 -- 172.21.15.64:0/2758645608 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf38004b70 con 0x559cf9e3ae70 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.989+0000 7fcf50517b80 10 client.25096 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.989+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559cf9bffc70 con 0x559cf9e3ae70 2024-05-08T21:56:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf25ffb700 1 -- 172.21.15.64:0/2758645608 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf38015070 con 0x559cf9e3ae70 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf25ffb700 1 -- 172.21.15.64:0/2758645608 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fcf38003680 con 0x559cf9e3ae70 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf50517b80 10 client.25096 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf50517b80 10 client.25096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf50517b80 20 client.25096 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e96280 0x559cf9e986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf50517b80 4 client.25096 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:56:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.990+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559cf9d72e60 con 0x559cf9e96280 2024-05-08T21:56:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.991+0000 7fcf418f8700 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e96280 0x559cf9e986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.993+0000 7fcf418f8700 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e96280 0x559cf9e986a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937ed38708e99cdc server_cookie=38c3e562a702b525 in_seq=0 out_seq=0 2024-05-08T21:56:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:10.993+0000 7fcf25ffb700 10 client.25096 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.008+0000 7fcf25ffb700 1 -- 172.21.15.64:0/2758645608 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559cf9d72e60 con 0x559cf9e96280 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.008+0000 7fcf25ffb700 10 client.25096 handle_command_reply: tid=0 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 client.25096 shutdown 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 msgr2=0x559cf9e38590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 0x559cf9e38590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf38005300 tx=0x7fcf38004610 comp rx=0 tx=0).stop 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 shutdown_connections 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559cf9e29a80 0x559cf9e38050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559cf9e96280 0x559cf9e986a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf9e393b0 0x559cf9e39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 --2- 172.21.15.64:0/2758645608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cf9e3ae70 0x559cf9e38590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 >> 172.21.15.64:0/2758645608 conn(0x559cf9d881b0 msgr2=0x559cf9e463d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf25ffb700 0 client.25096 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 shutdown_connections 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 1 -- 172.21.15.64:0/2758645608 wait complete. 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.009+0000 7fcf50517b80 20 client.25096 trim_cache size 0 max 16384 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:56:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.010+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcf280a1200 msgr2=0x7fcf280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.010+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcf280a1200 0x7fcf280a36c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf34007b40 tx=0x7fcf3400b040 comp rx=0 tx=0).stop 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.010+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1013c0 msgr2=0x7fcf3c110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.010+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1013c0 0x7fcf3c110240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf34007c70 tx=0x7fcf340095c0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 shutdown_connections 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcf280a1200 0x7fcf280a36c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf3c1013c0 0x7fcf3c110240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c100010 0x7fcf3c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 --2- 172.21.15.64:0/3746236781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c1009d0 0x7fcf3c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 >> 172.21.15.64:0/3746236781 conn(0x7fcf3c0fab20 msgr2=0x7fcf3c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.012+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 shutdown_connections 2024-05-08T21:56:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:11.013+0000 7fcf4435d700 1 -- 172.21.15.64:0/3746236781 wait complete. 2024-05-08T21:56:11.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 80.0 s, 4720.6 tps, lat 21.191 ms stddev 22.193 2024-05-08T21:56:11.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:11 smithi160 ceph-mon[46962]: pgmap v306: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 45 MiB/s wr, 1.91k op/s 2024-05-08T21:56:11.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:11 smithi186 ceph-mon[46619]: pgmap v306: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 45 MiB/s wr, 1.91k op/s 2024-05-08T21:56:11.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:11 smithi064 ceph-mon[42500]: pgmap v306: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 45 MiB/s wr, 1.91k op/s 2024-05-08T21:56:13.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:13 smithi160 ceph-mon[46962]: pgmap v307: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 1.23k op/s 2024-05-08T21:56:13.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:13 smithi186 ceph-mon[46619]: pgmap v307: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 1.23k op/s 2024-05-08T21:56:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:13 smithi064 ceph-mon[42500]: pgmap v307: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 1.23k op/s 2024-05-08T21:56:15.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:15 smithi160 ceph-mon[46962]: pgmap v308: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 962 op/s 2024-05-08T21:56:15.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:15 smithi186 ceph-mon[46619]: pgmap v308: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 962 op/s 2024-05-08T21:56:15.794 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:15 smithi064 ceph-mon[42500]: pgmap v308: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 962 op/s 2024-05-08T21:56:16.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 85.0 s, 3625.8 tps, lat 27.483 ms stddev 30.480 2024-05-08T21:56:17.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:17 smithi160 ceph-mon[46962]: pgmap v309: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 20 MiB/s wr, 893 op/s 2024-05-08T21:56:17.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:17 smithi186 ceph-mon[46619]: pgmap v309: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 20 MiB/s wr, 893 op/s 2024-05-08T21:56:17.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:17 smithi064 ceph-mon[42500]: pgmap v309: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 20 MiB/s wr, 893 op/s 2024-05-08T21:56:19.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:19 smithi160 ceph-mon[46962]: pgmap v310: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 971 op/s 2024-05-08T21:56:19.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:19 smithi186 ceph-mon[46619]: pgmap v310: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 971 op/s 2024-05-08T21:56:19.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:19 smithi064 ceph-mon[42500]: pgmap v310: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 971 op/s 2024-05-08T21:56:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 90.0 s, 3641.4 tps, lat 25.737 ms stddev 29.246 2024-05-08T21:56:21.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:21 smithi160 ceph-mon[46962]: pgmap v311: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 1.14k op/s 2024-05-08T21:56:21.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:21 smithi186 ceph-mon[46619]: pgmap v311: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 1.14k op/s 2024-05-08T21:56:21.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:21 smithi064 ceph-mon[42500]: pgmap v311: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 1.14k op/s 2024-05-08T21:56:23.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:23 smithi160 ceph-mon[46962]: pgmap v312: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2024-05-08T21:56:23.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:23 smithi186 ceph-mon[46619]: pgmap v312: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2024-05-08T21:56:23.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:23 smithi064 ceph-mon[42500]: pgmap v312: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.27k op/s 2024-05-08T21:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 -- 172.21.15.64:0/1471397960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40108730 msgr2=0x7f0e40108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 --2- 172.21.15.64:0/1471397960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40108730 0x7f0e40108b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009990 tx=0x7f0e3c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 -- 172.21.15.64:0/1471397960 shutdown_connections 2024-05-08T21:56:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 --2- 172.21.15.64:0/1471397960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40107540 0x7f0e40107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 --2- 172.21.15.64:0/1471397960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e401090d0 0x7f0e4010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 --2- 172.21.15.64:0/1471397960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40108730 0x7f0e40108b90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 -- 172.21.15.64:0/1471397960 >> 172.21.15.64:0/1471397960 conn(0x7f0e400764d0 msgr2=0x7f0e40078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 -- 172.21.15.64:0/1471397960 shutdown_connections 2024-05-08T21:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.194+0000 7f0e4830e700 1 -- 172.21.15.64:0/1471397960 wait complete. 2024-05-08T21:56:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.196+0000 7f0e4830e700 1 Processor -- start 2024-05-08T21:56:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 -- start start 2024-05-08T21:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40107540 0x7f0e4019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 0x7f0e4019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e401090d0 0x7f0e401a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e401a50e0 con 0x7f0e401090d0 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e401a5220 con 0x7f0e40108730 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e4830e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e401a53b0 con 0x7f0e40107540 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 0x7f0e4019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 0x7f0e4019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:58078/0 (socket says 172.21.15.64:58078) 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 -- 172.21.15.64:0/2430315776 learned_addr learned my addr 172.21.15.64:0/2430315776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e468ab700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e401090d0 0x7f0e401a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 -- 172.21.15.64:0/2430315776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40107540 msgr2=0x7f0e4019d5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40107540 0x7f0e4019d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 -- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e401090d0 msgr2=0x7f0e401a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.197+0000 7f0e458a9700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e401090d0 0x7f0e401a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.198+0000 7f0e458a9700 1 -- 172.21.15.64:0/2430315776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c0095f0 con 0x7f0e40108730 2024-05-08T21:56:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.198+0000 7f0e458a9700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 0x7f0e4019daf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00f690 tx=0x7f0e3c00f770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.198+0000 7f0e377fe700 1 -- 172.21.15.64:0/2430315776 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c01d070 con 0x7f0e40108730 2024-05-08T21:56:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.198+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e401a5630 con 0x7f0e40108730 2024-05-08T21:56:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.198+0000 7f0e377fe700 1 -- 172.21.15.64:0/2430315776 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e3c00fc90 con 0x7f0e40108730 2024-05-08T21:56:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.198+0000 7f0e377fe700 1 -- 172.21.15.64:0/2430315776 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c017650 con 0x7f0e40108730 2024-05-08T21:56:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.199+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e401a5ab0 con 0x7f0e40108730 2024-05-08T21:56:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.203+0000 7f0e544c8b80 1 Processor -- start 2024-05-08T21:56:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 -- start start 2024-05-08T21:56:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc65a80 0x5578dcc65ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc66cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc672d0 0x5578dcc67750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcbbd0f0 con 0x5578dcc672d0 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcbbaab0 con 0x5578dcc663e0 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcbb63a0 con 0x5578dcc65a80 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e460aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc66cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e460aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc66cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:58090/0 (socket says 172.21.15.64:58090) 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e460aa700 1 -- 172.21.15.64:0/4083002998 learned_addr learned my addr 172.21.15.64:0/4083002998 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.204+0000 7f0e468ab700 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc65a80 0x5578dcc65ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e460aa700 1 -- 172.21.15.64:0/4083002998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc65a80 msgr2=0x5578dcc65ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e460aa700 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc65a80 0x5578dcc65ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e460aa700 1 -- 172.21.15.64:0/4083002998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc672d0 msgr2=0x5578dcc67750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e460aa700 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc672d0 0x5578dcc67750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e460aa700 1 -- 172.21.15.64:0/4083002998 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578dca37350 con 0x5578dcc663e0 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e377fe700 1 -- 172.21.15.64:0/2430315776 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0e3c022470 con 0x7f0e40108730 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e460aa700 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc66cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009d50 tx=0x7f0e3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998b4bde2a1932dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e34da4700 1 -- 172.21.15.64:0/4083002998 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800e070 con 0x5578dcc663e0 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e34da4700 1 -- 172.21.15.64:0/4083002998 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e38009880 con 0x5578dcc663e0 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.205+0000 7f0e34da4700 1 -- 172.21.15.64:0/4083002998 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38011390 con 0x5578dcc663e0 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e377fe700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0e2c0a11a0 0x7f0e2c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e377fe700 1 -- 172.21.15.64:0/2430315776 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e3c0cde70 con 0x7f0e40108730 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e468ab700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0e2c0a11a0 0x7f0e2c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 -- 172.21.15.64:0/4083002998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 msgr2=0x5578dcc66cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc66cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009d50 tx=0x7f0e3800b330 comp rx=0 tx=0).stop 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 -- 172.21.15.64:0/4083002998 shutdown_connections 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc672d0 0x5578dcc67750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc65a80 0x5578dcc65ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/4083002998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc66cf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e544c8b80 1 -- 172.21.15.64:0/4083002998 >> 172.21.15.64:0/4083002998 conn(0x5578dcbc41b0 msgr2=0x5578dcc62440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.206+0000 7f0e468ab700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0e2c0a11a0 0x7f0e2c0a3660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000ba90 tx=0x7f0e30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 -- 172.21.15.64:0/4083002998 shutdown_connections 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 -- 172.21.15.64:0/4083002998 wait complete. 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 -- start start 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 Processor -- start 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc6f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc6fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcb52500 con 0x5578dcc65a80 2024-05-08T21:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcb521a0 con 0x5578dcc663e0 2024-05-08T21:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dca39640 con 0x5578dcc672d0 2024-05-08T21:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.207+0000 7f0e544c8b80 10 client.0 _ll_register_callbacks cb 0x5578dcb5bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc6fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc6fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:58106/0 (socket says 172.21.15.64:58106) 2024-05-08T21:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 -- 172.21.15.64:0/1727984149 learned_addr learned my addr 172.21.15.64:0/1727984149 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e460aa700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc6f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e468ab700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 -- 172.21.15.64:0/1727984149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 msgr2=0x5578dcc76e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc76e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 -- 172.21.15.64:0/1727984149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 msgr2=0x5578dcc6f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc6f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 -- 172.21.15.64:0/1727984149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e38009a00 con 0x5578dcc663e0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e460aa700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc6f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e458a9700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc6fe30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098e0 tx=0x7f0e3c09adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b402615d4085c35d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.208+0000 7f0e29ffb700 1 -- 172.21.15.64:0/1727984149 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c09aa50 con 0x5578dcc663e0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.209+0000 7f0e29ffb700 1 -- 172.21.15.64:0/1727984149 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e3c0cbe40 con 0x5578dcc663e0 2024-05-08T21:56:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.209+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578dcc357b0 con 0x5578dcc663e0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.209+0000 7f0e29ffb700 1 -- 172.21.15.64:0/1727984149 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e3c00fe00 con 0x5578dcc663e0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.209+0000 7f0e544c8b80 10 client.25163 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.209+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5578dc9e3b00 con 0x5578dcc663e0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.209+0000 7f0e29ffb700 1 -- 172.21.15.64:0/1727984149 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578dcc357b0 con 0x5578dcc663e0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e29ffb700 1 -- 172.21.15.64:0/1727984149 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f0e3c0a32c0 con 0x5578dcc663e0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e544c8b80 10 client.25163 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e544c8b80 10 client.25163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e544c8b80 20 client.25163 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcc7b410 0x5578dcc7d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e544c8b80 4 client.25163 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578dcbaee60 con 0x5578dcc7b410 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e460aa700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcc7b410 0x5578dcc7d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e460aa700 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcc7b410 0x5578dcc7d830 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0aba83b88df22c server_cookie=a18d0283db0eb769 in_seq=0 out_seq=0 2024-05-08T21:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.210+0000 7f0e29ffb700 10 client.25163 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e29ffb700 1 -- 172.21.15.64:0/1727984149 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578dcbaee60 con 0x5578dcc7b410 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e29ffb700 10 client.25163 handle_command_reply: tid=0 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 client.25163 shutdown 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 msgr2=0x5578dcc6fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc6fe30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098e0 tx=0x7f0e3c09adf0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 shutdown_connections 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcc7b410 0x5578dcc7d830 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc76e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc6f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/1727984149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc663e0 0x5578dcc6fe30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 >> 172.21.15.64:0/1727984149 conn(0x5578dcbc41b0 msgr2=0x5578dcc60860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e29ffb700 0 client.25163 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 shutdown_connections 2024-05-08T21:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.211+0000 7f0e544c8b80 1 -- 172.21.15.64:0/1727984149 wait complete. 2024-05-08T21:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.212+0000 7f0e544c8b80 20 client.25163 trim_cache size 0 max 16384 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.233+0000 7f0e544c8b80 1 Processor -- start 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 -- start start 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc663e0 0x5578dcc76e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc672d0 0x5578dcc77520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcc5b7c0 con 0x5578dcc65a80 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcc5b330 con 0x5578dcc672d0 2024-05-08T21:56:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcc70790 con 0x5578dcc663e0 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43104/0 (socket says 172.21.15.64:43104) 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 -- 172.21.15.64:0/2880627607 learned_addr learned my addr 172.21.15.64:0/2880627607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e458a9700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc663e0 0x5578dcc76e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e468ab700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc672d0 0x5578dcc77520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 -- 172.21.15.64:0/2880627607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc663e0 msgr2=0x5578dcc76e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc663e0 0x5578dcc76e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 -- 172.21.15.64:0/2880627607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc672d0 msgr2=0x5578dcc77520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc672d0 0x5578dcc77520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 -- 172.21.15.64:0/2880627607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578dcb7c500 con 0x5578dcc65a80 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e458a9700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc663e0 0x5578dcc76e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e468ab700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc672d0 0x5578dcc77520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.234+0000 7f0e460aa700 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc676b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009d20 tx=0x7f0e380096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6211ac36e43d5817 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e355a5700 1 -- 172.21.15.64:0/2880627607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800e040 con 0x5578dcc65a80 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e355a5700 1 -- 172.21.15.64:0/2880627607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e38028070 con 0x5578dcc65a80 2024-05-08T21:56:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e355a5700 1 -- 172.21.15.64:0/2880627607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3801b3f0 con 0x5578dcc65a80 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 -- 172.21.15.64:0/2880627607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 msgr2=0x5578dcc676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc676b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009d20 tx=0x7f0e380096e0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 -- 172.21.15.64:0/2880627607 shutdown_connections 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc672d0 0x5578dcc77520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc65a80 0x5578dcc676b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/2880627607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc663e0 0x5578dcc76e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.235+0000 7f0e544c8b80 1 -- 172.21.15.64:0/2880627607 >> 172.21.15.64:0/2880627607 conn(0x5578dcbc41b0 msgr2=0x5578dcc6ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 -- 172.21.15.64:0/2880627607 shutdown_connections 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 -- 172.21.15.64:0/2880627607 wait complete. 2024-05-08T21:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 -- start start 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 Processor -- start 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc65a80 0x5578dcc724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc663e0 0x5578dcc72a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc75890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcc75dd0 con 0x5578dcc663e0 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcc75f40 con 0x5578dcc65a80 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578dcc760b0 con 0x5578dcc672d0 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.236+0000 7f0e544c8b80 10 client.0 _ll_register_callbacks cb 0x5578dcb5bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.237+0000 7f0e468ab700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc75890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.237+0000 7f0e468ab700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc75890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42194/0 (socket says 172.21.15.64:42194) 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.237+0000 7f0e468ab700 1 -- 172.21.15.64:0/3428062901 learned_addr learned my addr 172.21.15.64:0/3428062901 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.237+0000 7f0e460aa700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc65a80 0x5578dcc724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.237+0000 7f0e458a9700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc663e0 0x5578dcc72a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e458a9700 1 -- 172.21.15.64:0/3428062901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 msgr2=0x5578dcc75890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e458a9700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc75890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e458a9700 1 -- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc65a80 msgr2=0x5578dcc724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e458a9700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc65a80 0x5578dcc724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e458a9700 1 -- 172.21.15.64:0/3428062901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578dca37350 con 0x5578dcc663e0 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e468ab700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc75890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e458a9700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc663e0 0x5578dcc72a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c097860 tx=0x7f0e3c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c44434eda4891 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e29ffb700 1 -- 172.21.15.64:0/3428062901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0b9070 con 0x5578dcc663e0 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.238+0000 7f0e29ffb700 1 -- 172.21.15.64:0/3428062901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e3c0cbe40 con 0x5578dcc663e0 2024-05-08T21:56:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.239+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578dcb0f210 con 0x5578dcc663e0 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.239+0000 7f0e29ffb700 1 -- 172.21.15.64:0/3428062901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e3c00fe00 con 0x5578dcc663e0 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.239+0000 7f0e544c8b80 10 client.15447 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.239+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5578dca3bc70 con 0x5578dcc663e0 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.239+0000 7f0e29ffb700 1 -- 172.21.15.64:0/3428062901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e3c07e020 con 0x5578dcc663e0 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.239+0000 7f0e29ffb700 1 -- 172.21.15.64:0/3428062901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f0e3c09a2c0 con 0x5578dcc663e0 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e544c8b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e544c8b80 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e544c8b80 20 client.15447 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcce2c00 0x5578dcce4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e544c8b80 4 client.15447 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578dca04c90 con 0x5578dcce2c00 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.240+0000 7f0e460aa700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcce2c00 0x5578dcce4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.241+0000 7f0e460aa700 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcce2c00 0x5578dcce4ff0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78a4327ee583ebb server_cookie=c728427cabac4a43 in_seq=0 out_seq=0 2024-05-08T21:56:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.241+0000 7f0e29ffb700 10 client.15447 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:24.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e29ffb700 1 -- 172.21.15.64:0/3428062901 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578dca04c90 con 0x5578dcce2c00 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e29ffb700 10 client.15447 handle_command_reply: tid=0 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 client.15447 shutdown 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc663e0 msgr2=0x5578dcc72a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc663e0 0x5578dcc72a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c097860 tx=0x7f0e3c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 shutdown_connections 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578dcc65a80 0x5578dcc724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578dcce2c00 0x5578dcce4ff0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578dcc672d0 0x5578dcc75890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 --2- 172.21.15.64:0/3428062901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578dcc663e0 0x5578dcc72a10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 >> 172.21.15.64:0/3428062901 conn(0x5578dcbc41b0 msgr2=0x5578dcc6a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.258+0000 7f0e29ffb700 0 client.15447 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:24.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.261+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 shutdown_connections 2024-05-08T21:56:24.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.261+0000 7f0e544c8b80 1 -- 172.21.15.64:0/3428062901 wait complete. 2024-05-08T21:56:24.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.261+0000 7f0e544c8b80 20 client.15447 trim_cache size 0 max 16384 2024-05-08T21:56:24.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:56:24.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:56:24.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:56:24.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0e2c0a11a0 msgr2=0x7f0e2c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0e2c0a11a0 0x7f0e2c0a3660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000ba90 tx=0x7f0e30008040 comp rx=0 tx=0).stop 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 msgr2=0x7f0e4019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 0x7f0e4019daf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00f690 tx=0x7f0e3c00f770 comp rx=0 tx=0).stop 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 shutdown_connections 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0e2c0a11a0 0x7f0e2c0a3660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e401090d0 0x7f0e401a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e40107540 0x7f0e4019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 --2- 172.21.15.64:0/2430315776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e40108730 0x7f0e4019daf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 >> 172.21.15.64:0/2430315776 conn(0x7f0e400764d0 msgr2=0x7f0e4010ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 shutdown_connections 2024-05-08T21:56:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:24.262+0000 7f0e4830e700 1 -- 172.21.15.64:0/2430315776 wait complete. 2024-05-08T21:56:25.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:25 smithi160 ceph-mon[46962]: pgmap v313: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 MiB/s wr, 1.89k op/s 2024-05-08T21:56:25.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:25 smithi186 ceph-mon[46619]: pgmap v313: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 MiB/s wr, 1.89k op/s 2024-05-08T21:56:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:25 smithi064 ceph-mon[42500]: pgmap v313: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 MiB/s wr, 1.89k op/s 2024-05-08T21:56:26.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 95.0 s, 3305.9 tps, lat 32.186 ms stddev 46.484 2024-05-08T21:56:26.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:26 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:56:26.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:26 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:56:26.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:26 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:56:27.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:27 smithi160 ceph-mon[46962]: pgmap v314: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 50 MiB/s wr, 1.99k op/s 2024-05-08T21:56:27.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:56:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:56:27.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:56:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:27 smithi186 ceph-mon[46619]: pgmap v314: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 50 MiB/s wr, 1.99k op/s 2024-05-08T21:56:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:56:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:56:27.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:56:27.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:27 smithi064 ceph-mon[42500]: pgmap v314: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 50 MiB/s wr, 1.99k op/s 2024-05-08T21:56:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:56:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:56:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:56:29.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:29 smithi186 ceph-mon[46619]: pgmap v315: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 2.06k op/s 2024-05-08T21:56:29.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:29 smithi064 ceph-mon[42500]: pgmap v315: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 2.06k op/s 2024-05-08T21:56:29.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:29 smithi160 ceph-mon[46962]: pgmap v315: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 2.06k op/s 2024-05-08T21:56:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 100.0 s, 3904.5 tps, lat 25.618 ms stddev 27.794 2024-05-08T21:56:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:31 smithi186 ceph-mon[46619]: pgmap v316: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 54 MiB/s wr, 2.10k op/s 2024-05-08T21:56:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:31 smithi064 ceph-mon[42500]: pgmap v316: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 54 MiB/s wr, 2.10k op/s 2024-05-08T21:56:31.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:31 smithi160 ceph-mon[46962]: pgmap v316: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 54 MiB/s wr, 2.10k op/s 2024-05-08T21:56:33.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:33 smithi186 ceph-mon[46619]: pgmap v317: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 MiB/s wr, 1.92k op/s 2024-05-08T21:56:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:33 smithi064 ceph-mon[42500]: pgmap v317: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 MiB/s wr, 1.92k op/s 2024-05-08T21:56:33.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:33 smithi160 ceph-mon[46962]: pgmap v317: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 MiB/s wr, 1.92k op/s 2024-05-08T21:56:35.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:35 smithi186 ceph-mon[46619]: pgmap v318: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 MiB/s wr, 1.77k op/s 2024-05-08T21:56:35.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:35 smithi064 ceph-mon[42500]: pgmap v318: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 MiB/s wr, 1.77k op/s 2024-05-08T21:56:35.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:35 smithi160 ceph-mon[46962]: pgmap v318: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 MiB/s wr, 1.77k op/s 2024-05-08T21:56:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 105.0 s, 4296.0 tps, lat 23.267 ms stddev 25.727 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.446+0000 7fb97158a700 1 -- 172.21.15.64:0/18698138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c104cd0 msgr2=0x7fb96c10b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.446+0000 7fb97158a700 1 --2- 172.21.15.64:0/18698138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c104cd0 0x7fb96c10b580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb96401c100 tx=0x7fb96401c410 comp rx=0 tx=0).stop 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.447+0000 7fb97158a700 1 -- 172.21.15.64:0/18698138 shutdown_connections 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.447+0000 7fb97158a700 1 --2- 172.21.15.64:0/18698138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c103140 0x7fb96c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.447+0000 7fb97158a700 1 --2- 172.21.15.64:0/18698138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c104cd0 0x7fb96c10b580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.447+0000 7fb97158a700 1 --2- 172.21.15.64:0/18698138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 0x7fb96c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.447+0000 7fb97158a700 1 -- 172.21.15.64:0/18698138 >> 172.21.15.64:0/18698138 conn(0x7fb96c0fe6b0 msgr2=0x7fb96c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.450+0000 7fb97158a700 1 -- 172.21.15.64:0/18698138 shutdown_connections 2024-05-08T21:56:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.450+0000 7fb97158a700 1 -- 172.21.15.64:0/18698138 wait complete. 2024-05-08T21:56:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.451+0000 7fb97158a700 1 Processor -- start 2024-05-08T21:56:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.451+0000 7fb97158a700 1 -- start start 2024-05-08T21:56:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb97158a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c103140 0x7fb96c06e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb97158a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 0x7fb96c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb97158a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c104cd0 0x7fb96c071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb97158a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c072310 con 0x7fb96c103140 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb97158a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c072480 con 0x7fb96c104cd0 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb97158a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c072640 con 0x7fb96c104330 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c103140 0x7fb96c06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 0x7fb96c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c103140 0x7fb96c06e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49350/0 (socket says 172.21.15.64:49350) 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96affd700 1 -- 172.21.15.64:0/1739269735 learned_addr learned my addr 172.21.15.64:0/1739269735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c104cd0 0x7fb96c071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96a7fc700 1 -- 172.21.15.64:0/1739269735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c104cd0 msgr2=0x7fb96c071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c104cd0 0x7fb96c071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96a7fc700 1 -- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c103140 msgr2=0x7fb96c06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c103140 0x7fb96c06e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.452+0000 7fb96a7fc700 1 -- 172.21.15.64:0/1739269735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96401ad10 con 0x7fb96c104330 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.453+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 0x7fb96c06ed40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb960006fd0 tx=0x7fb96000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.453+0000 7fb953fff700 1 -- 172.21.15.64:0/1739269735 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960011830 con 0x7fb96c104330 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.453+0000 7fb953fff700 1 -- 172.21.15.64:0/1739269735 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb960011e50 con 0x7fb96c104330 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.453+0000 7fb953fff700 1 -- 172.21.15.64:0/1739269735 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960010de0 con 0x7fb96c104330 2024-05-08T21:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.453+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c0728c0 con 0x7fb96c104330 2024-05-08T21:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.453+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c072d80 con 0x7fb96c104330 2024-05-08T21:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 Processor -- start 2024-05-08T21:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 -- start start 2024-05-08T21:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb953fff700 1 -- 172.21.15.64:0/1739269735 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb960010480 con 0x7fb96c104330 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d6cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d6dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311cc40f0 con 0x557311d6d3e0 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311cc1ab0 con 0x557311d6ca80 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.458+0000 7fb97d744b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311cbd3a0 con 0x557311d6e2d0 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d6cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d6cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:44484/0 (socket says 172.21.15.64:44484) 2024-05-08T21:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 -- 172.21.15.64:0/2035858473 learned_addr learned my addr 172.21.15.64:0/2035858473 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb953fff700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb9540a1320 0x7fb9540a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96affd700 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d6dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 -- 172.21.15.64:0/2035858473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 msgr2=0x557311d6e750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 -- 172.21.15.64:0/2035858473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 msgr2=0x557311d6dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d6dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb953fff700 1 -- 172.21.15.64:0/1739269735 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9600cc210 con 0x7fb96c104330 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 -- 172.21.15.64:0/2035858473 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557311b3e350 con 0x557311d6ca80 2024-05-08T21:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb9540a1320 0x7fb9540a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.459+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d6cea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb96401a560 tx=0x7fb96401a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50136bdc95d05bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb9517fa700 1 -- 172.21.15.64:0/2035858473 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96401f070 con 0x557311d6ca80 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb9540a1320 0x7fb9540a37e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb96403a000 tx=0x7fb964033000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb9517fa700 1 -- 172.21.15.64:0/2035858473 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb96401dec0 con 0x557311d6ca80 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb9517fa700 1 -- 172.21.15.64:0/2035858473 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96402e6b0 con 0x557311d6ca80 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb97d744b80 1 -- 172.21.15.64:0/2035858473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 msgr2=0x557311d6cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb97d744b80 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d6cea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb96401a560 tx=0x7fb96401a590 comp rx=0 tx=0).stop 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb97d744b80 1 -- 172.21.15.64:0/2035858473 shutdown_connections 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.460+0000 7fb97d744b80 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d6cea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 --2- 172.21.15.64:0/2035858473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d6dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 -- 172.21.15.64:0/2035858473 >> 172.21.15.64:0/2035858473 conn(0x557311ccb1b0 msgr2=0x557311d69440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 -- 172.21.15.64:0/2035858473 shutdown_connections 2024-05-08T21:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 -- 172.21.15.64:0/2035858473 wait complete. 2024-05-08T21:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 -- start start 2024-05-08T21:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.461+0000 7fb97d744b80 1 Processor -- start 2024-05-08T21:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 0x557311d76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6e2d0 0x557311d7deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311c59500 con 0x557311d6d3e0 2024-05-08T21:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311c591a0 con 0x557311d6e2d0 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311b40640 con 0x557311d6ca80 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.462+0000 7fb97d744b80 10 client.0 _ll_register_callbacks cb 0x557311c62ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 0x557311d76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 0x557311d76930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42186/0 (socket says 172.21.15.64:42186) 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 -- 172.21.15.64:0/215980007 learned_addr learned my addr 172.21.15.64:0/215980007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 -- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6e2d0 msgr2=0x557311d7deb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6e2d0 0x557311d7deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 -- 172.21.15.64:0/215980007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 msgr2=0x557311d76e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 -- 172.21.15.64:0/215980007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c10bed0 con 0x557311d6ca80 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb96affd700 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 0x557311d76930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0127f0 tx=0x7fb95c012bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5029f4eca4a92c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.463+0000 7fb94eff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.464+0000 7fb94eff5700 1 -- 172.21.15.64:0/215980007 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c017070 con 0x557311d6ca80 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.464+0000 7fb94eff5700 1 -- 172.21.15.64:0/215980007 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb95c00f640 con 0x557311d6ca80 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.464+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557311d3c7b0 con 0x557311d6ca80 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.464+0000 7fb94eff5700 1 -- 172.21.15.64:0/215980007 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb95c0099a0 con 0x557311d6ca80 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.464+0000 7fb97d744b80 10 client.25114 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.464+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557311aeab00 con 0x557311d6ca80 2024-05-08T21:56:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb94eff5700 1 -- 172.21.15.64:0/215980007 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557311d3c7b0 con 0x557311d6ca80 2024-05-08T21:56:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb94eff5700 1 -- 172.21.15.64:0/215980007 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fb95c010a20 con 0x557311d6ca80 2024-05-08T21:56:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb97d744b80 10 client.25114 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb97d744b80 10 client.25114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb97d744b80 20 client.25114 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb97d744b80 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311d93860 0x557311d95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb97d744b80 4 client.25114 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557311cb5e60 con 0x557311d93860 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311d93860 0x557311d95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb94eff5700 10 client.25114 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.467+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311d93860 0x557311d95c80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3288c38a1798db06 server_cookie=cedddc1fd3c965ef in_seq=0 out_seq=0 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb94eff5700 1 -- 172.21.15.64:0/215980007 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557311cb5e60 con 0x557311d93860 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb94eff5700 10 client.25114 handle_command_reply: tid=0 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 client.25114 shutdown 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 msgr2=0x557311d76930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 0x557311d76930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0127f0 tx=0x7fb95c012bb0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 shutdown_connections 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6e2d0 0x557311d7deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311d93860 0x557311d95c80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6ca80 0x557311d76930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 --2- 172.21.15.64:0/215980007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d76e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.468+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 >> 172.21.15.64:0/215980007 conn(0x557311ccb1b0 msgr2=0x557311d67860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.469+0000 7fb94eff5700 0 client.25114 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.469+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 shutdown_connections 2024-05-08T21:56:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.469+0000 7fb97d744b80 1 -- 172.21.15.64:0/215980007 wait complete. 2024-05-08T21:56:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.471+0000 7fb97d744b80 20 client.25114 trim_cache size 0 max 16384 2024-05-08T21:56:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.489+0000 7fb97d744b80 1 Processor -- start 2024-05-08T21:56:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.489+0000 7fb97d744b80 1 -- start start 2024-05-08T21:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d7de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb97d744b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311d62330 con 0x557311d6d3e0 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb97d744b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311d90040 con 0x557311d6ca80 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb97d744b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311d6c5e0 con 0x557311d6e2d0 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d8f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:44496/0 (socket says 172.21.15.64:44496) 2024-05-08T21:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 -- 172.21.15.64:0/3098476817 learned_addr learned my addr 172.21.15.64:0/3098476817 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d7de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 -- 172.21.15.64:0/3098476817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 msgr2=0x557311d6e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 -- 172.21.15.64:0/3098476817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 msgr2=0x557311d7de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d7de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.490+0000 7fb96affd700 1 -- 172.21.15.64:0/3098476817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557311c83500 con 0x557311d6ca80 2024-05-08T21:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.491+0000 7fb96affd700 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d8f570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009410 tx=0x7fb95c00f430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed1b3f0d7b309fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.491+0000 7fb951ffb700 1 -- 172.21.15.64:0/3098476817 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c017040 con 0x557311d6ca80 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.491+0000 7fb951ffb700 1 -- 172.21.15.64:0/3098476817 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb95c01c420 con 0x557311d6ca80 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.491+0000 7fb951ffb700 1 -- 172.21.15.64:0/3098476817 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c010430 con 0x557311d6ca80 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 -- 172.21.15.64:0/3098476817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 msgr2=0x557311d8f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d8f570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009410 tx=0x7fb95c00f430 comp rx=0 tx=0).stop 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 -- 172.21.15.64:0/3098476817 shutdown_connections 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6ca80 0x557311d8f570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d6e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 --2- 172.21.15.64:0/3098476817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6d3e0 0x557311d7de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 -- 172.21.15.64:0/3098476817 >> 172.21.15.64:0/3098476817 conn(0x557311ccb1b0 msgr2=0x557311d748a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 -- 172.21.15.64:0/3098476817 shutdown_connections 2024-05-08T21:56:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 -- 172.21.15.64:0/3098476817 wait complete. 2024-05-08T21:56:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 -- start start 2024-05-08T21:56:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.492+0000 7fb97d744b80 1 Processor -- start 2024-05-08T21:56:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6ca80 0x557311d7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6d3e0 0x557311d7b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311d7caa0 con 0x557311d6ca80 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311d7cc10 con 0x557311d6d3e0 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311d7cdd0 con 0x557311d6e2d0 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb97d744b80 10 client.0 _ll_register_callbacks cb 0x557311c62ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb96b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb96b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d7c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42202/0 (socket says 172.21.15.64:42202) 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb96b7fe700 1 -- 172.21.15.64:0/254051174 learned_addr learned my addr 172.21.15.64:0/254051174 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.493+0000 7fb96a7fc700 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6d3e0 0x557311d7b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb96b7fe700 1 -- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6d3e0 msgr2=0x557311d7b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6d3e0 0x557311d7b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb96b7fe700 1 -- 172.21.15.64:0/254051174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6ca80 msgr2=0x557311d7b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6ca80 0x557311d7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb96b7fe700 1 -- 172.21.15.64:0/254051174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557311aeab00 con 0x557311d6e2d0 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb96b7fe700 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d7c560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb964018c10 tx=0x7fb96401a9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d574aeb56924cc68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb94eff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.494+0000 7fb94eff5700 1 -- 172.21.15.64:0/254051174 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96401f040 con 0x557311d6e2d0 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.495+0000 7fb94eff5700 1 -- 172.21.15.64:0/254051174 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb96402e860 con 0x557311d6e2d0 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.495+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557311c16210 con 0x557311d6e2d0 2024-05-08T21:56:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.495+0000 7fb94eff5700 1 -- 172.21.15.64:0/254051174 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb96402ea20 con 0x557311d6e2d0 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.495+0000 7fb97d744b80 10 client.25120 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.495+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557311b42c70 con 0x557311d6e2d0 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.495+0000 7fb94eff5700 1 -- 172.21.15.64:0/254051174 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb964052030 con 0x557311d6e2d0 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.496+0000 7fb94eff5700 1 -- 172.21.15.64:0/254051174 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fb964038440 con 0x557311d6e2d0 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.496+0000 7fb97d744b80 10 client.25120 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.496+0000 7fb97d744b80 10 client.25120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.496+0000 7fb97d744b80 20 client.25120 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.497+0000 7fb97d744b80 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311de9e10 0x557311dec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.497+0000 7fb97d744b80 4 client.25120 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.497+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557311b0bc90 con 0x557311de9e10 2024-05-08T21:56:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.497+0000 7fb96affd700 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311de9e10 0x557311dec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.497+0000 7fb96affd700 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311de9e10 0x557311dec200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da06961964b9e000 server_cookie=cfc2a70929c14933 in_seq=0 out_seq=0 2024-05-08T21:56:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.497+0000 7fb94eff5700 10 client.25120 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.511+0000 7fb94eff5700 1 -- 172.21.15.64:0/254051174 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557311b0bc90 con 0x557311de9e10 2024-05-08T21:56:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.511+0000 7fb94eff5700 10 client.25120 handle_command_reply: tid=0 2024-05-08T21:56:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.511+0000 7fb97d744b80 1 client.25120 shutdown 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.511+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 msgr2=0x557311d7c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.511+0000 7fb97d744b80 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d7c560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb964018c10 tx=0x7fb96401a9f0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 shutdown_connections 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x557311de9e10 0x557311dec200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557311d6e2d0 0x557311d7c560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557311d6ca80 0x557311d7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb94eff5700 0 client.25120 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 --2- 172.21.15.64:0/254051174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557311d6d3e0 0x557311d7b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 >> 172.21.15.64:0/254051174 conn(0x557311ccb1b0 msgr2=0x557311d9a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 shutdown_connections 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 1 -- 172.21.15.64:0/254051174 wait complete. 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.512+0000 7fb97d744b80 20 client.25120 trim_cache size 0 max 16384 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:56:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:56:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:56:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.513+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb9540a1320 msgr2=0x7fb9540a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.513+0000 7fb97158a700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb9540a1320 0x7fb9540a37e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb96403a000 tx=0x7fb964033000 comp rx=0 tx=0).stop 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.513+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 msgr2=0x7fb96c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.513+0000 7fb97158a700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 0x7fb96c06ed40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb960006fd0 tx=0x7fb96000b730 comp rx=0 tx=0).stop 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 shutdown_connections 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb9540a1320 0x7fb9540a37e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c104cd0 0x7fb96c071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb96c103140 0x7fb96c06e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 --2- 172.21.15.64:0/1739269735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb96c104330 0x7fb96c06ed40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 >> 172.21.15.64:0/1739269735 conn(0x7fb96c0fe6b0 msgr2=0x7fb96c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 shutdown_connections 2024-05-08T21:56:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:37.514+0000 7fb97158a700 1 -- 172.21.15.64:0/1739269735 wait complete. 2024-05-08T21:56:37.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:37 smithi186 ceph-mon[46619]: pgmap v319: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-05-08T21:56:37.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:37 smithi064 ceph-mon[42500]: pgmap v319: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-05-08T21:56:37.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:37 smithi160 ceph-mon[46962]: pgmap v319: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-05-08T21:56:39.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:39 smithi186 ceph-mon[46619]: pgmap v320: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 840 op/s 2024-05-08T21:56:39.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:39 smithi064 ceph-mon[42500]: pgmap v320: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 840 op/s 2024-05-08T21:56:39.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:39 smithi160 ceph-mon[46962]: pgmap v320: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 840 op/s 2024-05-08T21:56:41.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 110.0 s, 3596.2 tps, lat 27.732 ms stddev 31.515 2024-05-08T21:56:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:41 smithi186 ceph-mon[46619]: pgmap v321: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 889 op/s 2024-05-08T21:56:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:41 smithi064 ceph-mon[42500]: pgmap v321: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 889 op/s 2024-05-08T21:56:41.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:41 smithi160 ceph-mon[46962]: pgmap v321: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 889 op/s 2024-05-08T21:56:43.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:43 smithi186 ceph-mon[46619]: pgmap v322: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 828 op/s 2024-05-08T21:56:43.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:43 smithi064 ceph-mon[42500]: pgmap v322: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 828 op/s 2024-05-08T21:56:43.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:43 smithi160 ceph-mon[46962]: pgmap v322: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 828 op/s 2024-05-08T21:56:45.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:45 smithi186 ceph-mon[46619]: pgmap v323: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2024-05-08T21:56:45.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:45 smithi064 ceph-mon[42500]: pgmap v323: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2024-05-08T21:56:45.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:45 smithi160 ceph-mon[46962]: pgmap v323: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2024-05-08T21:56:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 115.0 s, 3632.0 tps, lat 27.623 ms stddev 30.097 2024-05-08T21:56:47.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:47 smithi186 ceph-mon[46619]: pgmap v324: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 904 op/s 2024-05-08T21:56:47.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:47 smithi064 ceph-mon[42500]: pgmap v324: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 904 op/s 2024-05-08T21:56:47.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:47 smithi160 ceph-mon[46962]: pgmap v324: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 904 op/s 2024-05-08T21:56:49.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:49 smithi064 ceph-mon[42500]: pgmap v325: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 988 op/s 2024-05-08T21:56:49.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:49 smithi160 ceph-mon[46962]: pgmap v325: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 988 op/s 2024-05-08T21:56:49.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:49 smithi186 ceph-mon[46619]: pgmap v325: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 988 op/s 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 -- 172.21.15.64:0/3887137552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4107540 msgr2=0x7fcdd4107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/3887137552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4107540 0x7fcdd4107960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40099c0 tx=0x7fcdc4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 -- 172.21.15.64:0/3887137552 shutdown_connections 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/3887137552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4107540 0x7fcdd4107960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/3887137552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd41090d0 0x7fcdd410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/3887137552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4108730 0x7fcdd4108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 -- 172.21.15.64:0/3887137552 >> 172.21.15.64:0/3887137552 conn(0x7fcdd40764d0 msgr2=0x7fcdd4078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 -- 172.21.15.64:0/3887137552 shutdown_connections 2024-05-08T21:56:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.683+0000 7fcdd8a07700 1 -- 172.21.15.64:0/3887137552 wait complete. 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 Processor -- start 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 -- start start 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4107540 0x7fcdd419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4108730 0x7fcdd419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd41090d0 0x7fcdd41a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41a5020 con 0x7fcdd41090d0 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41a5160 con 0x7fcdd4108730 2024-05-08T21:56:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd8a07700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41a52f0 con 0x7fcdd4107540 2024-05-08T21:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4108730 0x7fcdd419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4108730 0x7fcdd419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:52646/0 (socket says 172.21.15.64:52646) 2024-05-08T21:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.684+0000 7fcdd1d9b700 1 -- 172.21.15.64:0/227924196 learned_addr learned my addr 172.21.15.64:0/227924196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.685+0000 7fcdd259c700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4107540 0x7fcdd419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.685+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd41090d0 0x7fcdd41a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdd259c700 1 -- 172.21.15.64:0/227924196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4108730 msgr2=0x7fcdd419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdd259c700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4108730 0x7fcdd419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdd259c700 1 -- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd41090d0 msgr2=0x7fcdd41a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdd259c700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd41090d0 0x7fcdd41a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdd259c700 1 -- 172.21.15.64:0/227924196 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40095f0 con 0x7fcdd4107540 2024-05-08T21:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdd259c700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4107540 0x7fcdd419d4f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40098e0 tx=0x7fcdc400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.687+0000 7fcdcb7fe700 1 -- 172.21.15.64:0/227924196 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc401d070 con 0x7fcdd4107540 2024-05-08T21:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.688+0000 7fcdcb7fe700 1 -- 172.21.15.64:0/227924196 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdc400fe70 con 0x7fcdd4107540 2024-05-08T21:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.688+0000 7fcdcb7fe700 1 -- 172.21.15.64:0/227924196 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4004ba0 con 0x7fcdd4107540 2024-05-08T21:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.688+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd41a5570 con 0x7fcdd4107540 2024-05-08T21:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.689+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd41a5940 con 0x7fcdd4107540 2024-05-08T21:56:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.693+0000 7fcdcb7fe700 1 -- 172.21.15.64:0/227924196 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcdc4004d00 con 0x7fcdd4107540 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 Processor -- start 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcdcb7fe700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdb80a1220 0x7fcdb80a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcdcb7fe700 1 -- 172.21.15.64:0/227924196 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdc40cd170 con 0x7fcdd4107540 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 -- start start 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 0x55e6e7f34cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7e8b0f0 con 0x55e6e7f343e0 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7e88ab0 con 0x55e6e7f352d0 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7e843a0 con 0x55e6e7f33a80 2024-05-08T21:56:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdb80a1220 0x7fcdb80a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcdd1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 0x55e6e7f34cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcdd2d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.694+0000 7fcdd1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 0x55e6e7f34cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49814/0 (socket says 172.21.15.64:49814) 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 -- 172.21.15.64:0/1886114403 learned_addr learned my addr 172.21.15.64:0/1886114403 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd259c700 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 -- 172.21.15.64:0/1886114403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f33a80 msgr2=0x55e6e7f33ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 -- 172.21.15.64:0/1886114403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 msgr2=0x55e6e7f35750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f35750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 -- 172.21.15.64:0/1886114403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6e7d05350 con 0x55e6e7f343e0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 0x55e6e7f34cf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e50 tx=0x7fcdc0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210a5419d608d34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdb80a1220 0x7fcdb80a36e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e6e7f34ab0 tx=0x7fcdcc0092f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdc8ff9700 1 -- 172.21.15.64:0/1886114403 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0004a30 con 0x55e6e7f343e0 2024-05-08T21:56:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdc8ff9700 1 -- 172.21.15.64:0/1886114403 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdc00054e0 con 0x55e6e7f343e0 2024-05-08T21:56:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.695+0000 7fcdc8ff9700 1 -- 172.21.15.64:0/1886114403 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0019930 con 0x55e6e7f343e0 2024-05-08T21:56:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.696+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1886114403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 msgr2=0x55e6e7f34cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.696+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 0x55e6e7f34cf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e50 tx=0x7fcdc0011720 comp rx=0 tx=0).stop 2024-05-08T21:56:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1886114403 shutdown_connections 2024-05-08T21:56:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1886114403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f343e0 0x55e6e7f34cf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1886114403 >> 172.21.15.64:0/1886114403 conn(0x55e6e7e921b0 msgr2=0x55e6e7f30440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1886114403 shutdown_connections 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.699+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1886114403 wait complete. 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 -- start start 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 Processor -- start 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f3d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f3dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f44e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7e20500 con 0x55e6e7f352d0 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7e201a0 con 0x55e6e7f33a80 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7d07640 con 0x55e6e7f343e0 2024-05-08T21:56:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcde4bc1b80 10 client.0 _ll_register_callbacks cb 0x55e6e7e29ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcdd259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f3d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcdd259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f3d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:52660/0 (socket says 172.21.15.64:52660) 2024-05-08T21:56:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcdd259c700 1 -- 172.21.15.64:0/1330268473 learned_addr learned my addr 172.21.15.64:0/1330268473 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.700+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f44e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f3dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd259c700 1 -- 172.21.15.64:0/1330268473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 msgr2=0x55e6e7f3dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd259c700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f3dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd259c700 1 -- 172.21.15.64:0/1330268473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 msgr2=0x55e6e7f44e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd259c700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f44e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd259c700 1 -- 172.21.15.64:0/1330268473 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc0009a80 con 0x55e6e7f33a80 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdd259c700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f3d8a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcdc409a590 tx=0x7fcdc409a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb58d1f29b1fdd33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.701+0000 7fcdb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.702+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/1330268473 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc409ad60 con 0x55e6e7f33a80 2024-05-08T21:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.702+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/1330268473 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdc409a600 con 0x55e6e7f33a80 2024-05-08T21:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.702+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6e7ddd210 con 0x55e6e7f33a80 2024-05-08T21:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.702+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/1330268473 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdc409e430 con 0x55e6e7f33a80 2024-05-08T21:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.702+0000 7fcde4bc1b80 10 client.25196 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.702+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e6e7cb1b00 con 0x55e6e7f33a80 2024-05-08T21:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/1330268473 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6e7ddd210 con 0x55e6e7f33a80 2024-05-08T21:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/1330268473 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fcdc4099bc0 con 0x55e6e7f33a80 2024-05-08T21:56:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcde4bc1b80 10 client.25196 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcde4bc1b80 10 client.25196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcde4bc1b80 20 client.25196 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7f49410 0x55e6e7f4b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcde4bc1b80 4 client.25196 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.703+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6e7f037b0 con 0x55e6e7f49410 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.704+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7f49410 0x55e6e7f4b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.704+0000 7fcdb5ffb700 10 client.25196 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.704+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7f49410 0x55e6e7f4b830 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd51be9d4e2f2589 server_cookie=6c3b20ff0afbbdf9 in_seq=0 out_seq=0 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/1330268473 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6e7f037b0 con 0x55e6e7f49410 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcdb5ffb700 10 client.25196 handle_command_reply: tid=0 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 client.25196 shutdown 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 msgr2=0x55e6e7f3d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f3d8a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcdc409a590 tx=0x7fcdc409a270 comp rx=0 tx=0).stop 2024-05-08T21:56:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 shutdown_connections 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f3d8a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7f49410 0x55e6e7f4b830 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/1330268473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f3dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcdb5ffb700 0 client.25196 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 >> 172.21.15.64:0/1330268473 conn(0x55e6e7e921b0 msgr2=0x55e6e7f2e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.705+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 shutdown_connections 2024-05-08T21:56:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.706+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/1330268473 wait complete. 2024-05-08T21:56:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.706+0000 7fcde4bc1b80 20 client.25196 trim_cache size 0 max 16384 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 Processor -- start 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 -- start start 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f44de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7f34b80 con 0x55e6e7f33a80 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7f297c0 con 0x55e6e7f352d0 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7f2f930 con 0x55e6e7f343e0 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49834/0 (socket says 172.21.15.64:49834) 2024-05-08T21:56:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 -- 172.21.15.64:0/3893344963 learned_addr learned my addr 172.21.15.64:0/3893344963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f44de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 -- 172.21.15.64:0/3893344963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 msgr2=0x55e6e7f44de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f44de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 -- 172.21.15.64:0/3893344963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 msgr2=0x55e6e7f356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f356b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd259c700 1 -- 172.21.15.64:0/3893344963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6e7e4a500 con 0x55e6e7f33a80 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.730+0000 7fcdd1d9b700 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f44de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.731+0000 7fcdd259c700 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4098950 tx=0x7fcdc40a2620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910d8ad3dbf6fd9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.731+0000 7fcdc97fa700 1 -- 172.21.15.64:0/3893344963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc409ad60 con 0x55e6e7f33a80 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.731+0000 7fcdc97fa700 1 -- 172.21.15.64:0/3893344963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdc409a600 con 0x55e6e7f33a80 2024-05-08T21:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.731+0000 7fcdc97fa700 1 -- 172.21.15.64:0/3893344963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40b8b60 con 0x55e6e7f33a80 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/3893344963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 msgr2=0x55e6e7f33e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4098950 tx=0x7fcdc40a2620 comp rx=0 tx=0).stop 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/3893344963 shutdown_connections 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f352d0 0x55e6e7f356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f33a80 0x55e6e7f33e60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/3893344963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f44de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/3893344963 >> 172.21.15.64:0/3893344963 conn(0x55e6e7e921b0 msgr2=0x55e6e7f3cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/3893344963 shutdown_connections 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.732+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/3893344963 wait complete. 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 -- start start 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 Processor -- start 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f424c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7f43f00 con 0x55e6e7f352d0 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7f44070 con 0x55e6e7f33a80 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e7f44230 con 0x55e6e7f343e0 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcde4bc1b80 10 client.0 _ll_register_callbacks cb 0x55e6e7e29ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.733+0000 7fcdd1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f42a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56726/0 (socket says 172.21.15.64:56726) 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd1d9b700 1 -- 172.21.15.64:0/2494524373 learned_addr learned my addr 172.21.15.64:0/2494524373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd259c700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 -- 172.21.15.64:0/2494524373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 msgr2=0x55e6e7f42a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f42a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 -- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 msgr2=0x55e6e7f424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 -- 172.21.15.64:0/2494524373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6e7d05350 con 0x55e6e7f352d0 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdd2d9d700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f439c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007730 tx=0x7fcdcc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feb8dc8e3a69226 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.734+0000 7fcdb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/2494524373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc021070 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/2494524373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdcc016e80 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdc00054e0 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/2494524373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdcc01a610 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcde4bc1b80 10 client.15477 fetch_fsmap learned FSMap version 33 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e6e7cb1b00 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.735+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/2494524373 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdc00054e0 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/2494524373 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fcdcc02e070 con 0x55e6e7f352d0 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcde4bc1b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcde4bc1b80 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:56:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcde4bc1b80 20 client.15477 populate_metadata read hostname 'smithi064' 2024-05-08T21:56:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7fa02e0 0x55e6e7fa2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:56:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcde4bc1b80 4 client.15477 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:56:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e6e7e7ce60 con 0x55e6e7fa02e0 2024-05-08T21:56:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcdd259c700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7fa02e0 0x55e6e7fa2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:56:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcdb5ffb700 10 client.15477 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.736+0000 7fcdd259c700 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7fa02e0 0x55e6e7fa2700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8f7c208c023a59 server_cookie=322623b6034d2b59 in_seq=0 out_seq=0 2024-05-08T21:56:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcdb5ffb700 1 -- 172.21.15.64:0/2494524373 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e6e7e7ce60 con 0x55e6e7fa02e0 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcdb5ffb700 10 client.15477 handle_command_reply: tid=0 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 client.15477 shutdown 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 msgr2=0x55e6e7f439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f439c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007730 tx=0x7fcdcc009040 comp rx=0 tx=0).stop 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 shutdown_connections 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e6e7f33a80 0x55e6e7f424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e6e7fa02e0 0x55e6e7fa2700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6e7f352d0 0x55e6e7f439c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 --2- 172.21.15.64:0/2494524373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e6e7f343e0 0x55e6e7f42a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 >> 172.21.15.64:0/2494524373 conn(0x55e6e7e921b0 msgr2=0x55e6e7f38510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.751+0000 7fcdb5ffb700 0 client.15477 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:56:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.752+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 shutdown_connections 2024-05-08T21:56:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.752+0000 7fcde4bc1b80 1 -- 172.21.15.64:0/2494524373 wait complete. 2024-05-08T21:56:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.752+0000 7fcde4bc1b80 20 client.15477 trim_cache size 0 max 16384 2024-05-08T21:56:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:56:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:56:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:56:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:56:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdb80a1220 msgr2=0x7fcdb80a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdb80a1220 0x7fcdb80a36e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e6e7f34ab0 tx=0x7fcdcc0092f0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4107540 msgr2=0x7fcdd419d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4107540 0x7fcdd419d4f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40098e0 tx=0x7fcdc400fa00 comp rx=0 tx=0).stop 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 shutdown_connections 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcdb80a1220 0x7fcdb80a36e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdd41090d0 0x7fcdd41a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdd4107540 0x7fcdd419d4f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 --2- 172.21.15.64:0/227924196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdd4108730 0x7fcdd419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 >> 172.21.15.64:0/227924196 conn(0x7fcdd40764d0 msgr2=0x7fcdd410bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 shutdown_connections 2024-05-08T21:56:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:56:50.753+0000 7fcdd8a07700 1 -- 172.21.15.64:0/227924196 wait complete. 2024-05-08T21:56:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 120.0 s, 2615.3 tps, lat 37.974 ms stddev 62.614 2024-05-08T21:56:51.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:51 smithi064 ceph-mon[42500]: pgmap v326: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 MiB/s wr, 1.64k op/s 2024-05-08T21:56:51.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:51 smithi160 ceph-mon[46962]: pgmap v326: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 MiB/s wr, 1.64k op/s 2024-05-08T21:56:51.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:51 smithi186 ceph-mon[46619]: pgmap v326: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 MiB/s wr, 1.64k op/s 2024-05-08T21:56:53.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:53 smithi064 ceph-mon[42500]: pgmap v327: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 1.94k op/s 2024-05-08T21:56:53.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:53 smithi160 ceph-mon[46962]: pgmap v327: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 1.94k op/s 2024-05-08T21:56:53.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:53 smithi186 ceph-mon[46619]: pgmap v327: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 1.94k op/s 2024-05-08T21:56:55.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:55 smithi064 ceph-mon[42500]: pgmap v328: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 54 MiB/s wr, 2.09k op/s 2024-05-08T21:56:55.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:55 smithi160 ceph-mon[46962]: pgmap v328: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 54 MiB/s wr, 2.09k op/s 2024-05-08T21:56:55.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:55 smithi186 ceph-mon[46619]: pgmap v328: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 54 MiB/s wr, 2.09k op/s 2024-05-08T21:56:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 125.0 s, 3735.5 tps, lat 26.908 ms stddev 29.919 2024-05-08T21:56:57.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:57 smithi064 ceph-mon[42500]: pgmap v329: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-08T21:56:57.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:57 smithi160 ceph-mon[46962]: pgmap v329: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-08T21:56:57.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:57 smithi186 ceph-mon[46619]: pgmap v329: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-08T21:56:59.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:56:59 smithi064 ceph-mon[42500]: pgmap v330: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 MiB/s wr, 1.92k op/s 2024-05-08T21:56:59.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:56:59 smithi160 ceph-mon[46962]: pgmap v330: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 MiB/s wr, 1.92k op/s 2024-05-08T21:56:59.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:56:59 smithi186 ceph-mon[46619]: pgmap v330: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 MiB/s wr, 1.92k op/s 2024-05-08T21:57:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 130.0 s, 3519.2 tps, lat 28.421 ms stddev 30.724 2024-05-08T21:57:01.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:01 smithi064 ceph-mon[42500]: pgmap v331: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 MiB/s wr, 1.96k op/s 2024-05-08T21:57:01.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:01 smithi160 ceph-mon[46962]: pgmap v331: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 MiB/s wr, 1.96k op/s 2024-05-08T21:57:01.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:01 smithi186 ceph-mon[46619]: pgmap v331: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 MiB/s wr, 1.96k op/s 2024-05-08T21:57:03.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:03 smithi064 ceph-mon[42500]: pgmap v332: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 1.38k op/s 2024-05-08T21:57:03.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:03 smithi160 ceph-mon[46962]: pgmap v332: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 1.38k op/s 2024-05-08T21:57:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:03 smithi186 ceph-mon[46619]: pgmap v332: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 1.38k op/s 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 -- 172.21.15.64:0/1522176321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e0107550 msgr2=0x7ff5e0107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/1522176321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e0107550 0x7ff5e0107970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800b0d0 tx=0x7ff5d800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 -- 172.21.15.64:0/1522176321 shutdown_connections 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/1522176321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e01090e0 0x7ff5e010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/1522176321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e0107550 0x7ff5e0107970 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/1522176321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e0108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 -- 172.21.15.64:0/1522176321 >> 172.21.15.64:0/1522176321 conn(0x7ff5e0076510 msgr2=0x7ff5e0078930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 -- 172.21.15.64:0/1522176321 shutdown_connections 2024-05-08T21:57:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.937+0000 7ff5e6dac700 1 -- 172.21.15.64:0/1522176321 wait complete. 2024-05-08T21:57:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 Processor -- start 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 -- start start 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e0107550 0x7ff5e019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 0x7ff5e01a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e01a5190 con 0x7ff5e01090e0 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e01a52d0 con 0x7ff5e0108740 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.938+0000 7ff5e6dac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e01a5460 con 0x7ff5e0107550 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5e5349700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 0x7ff5e01a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e019dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:34822/0 (socket says 172.21.15.64:34822) 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5e5349700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 0x7ff5e01a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53062/0 (socket says 172.21.15.64:53062) 2024-05-08T21:57:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 -- 172.21.15.64:0/2004081981 learned_addr learned my addr 172.21.15.64:0/2004081981 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e0107550 0x7ff5e019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 -- 172.21.15.64:0/2004081981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e0107550 msgr2=0x7ff5e019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e0107550 0x7ff5e019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 -- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 msgr2=0x7ff5e01a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 0x7ff5e01a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 -- 172.21.15.64:0/2004081981 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d8009d00 con 0x7ff5e0108740 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5e5349700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 0x7ff5e01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5dffff700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e019dc40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000e9a0 tx=0x7ff5d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5ddffb700 1 -- 172.21.15.64:0/2004081981 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0009790 con 0x7ff5e0108740 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5ddffb700 1 -- 172.21.15.64:0/2004081981 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d0010430 con 0x7ff5e0108740 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5ddffb700 1 -- 172.21.15.64:0/2004081981 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0018680 con 0x7ff5e0108740 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.939+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e01a5740 con 0x7ff5e0108740 2024-05-08T21:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.940+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e01a5c00 con 0x7ff5e0108740 2024-05-08T21:57:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.943+0000 7ff5ddffb700 1 -- 172.21.15.64:0/2004081981 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff5d00105a0 con 0x7ff5e0108740 2024-05-08T21:57:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.945+0000 7ff5f2f66b80 1 Processor -- start 2024-05-08T21:57:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.945+0000 7ff5f2f66b80 1 -- start start 2024-05-08T21:57:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.945+0000 7ff5ddffb700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff5c80a1120 0x7ff5c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5ddffb700 1 -- 172.21.15.64:0/2004081981 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff5d0014070 con 0x7ff5e0108740 2024-05-08T21:57:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e7075750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6fcb0f0 con 0x5623e70752d0 2024-05-08T21:57:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6fc8ab0 con 0x5623e70743e0 2024-05-08T21:57:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.946+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6fc43a0 con 0x5623e7073a80 2024-05-08T21:57:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.947+0000 7ff5e5349700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff5c80a1120 0x7ff5c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.947+0000 7ff5e4b48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.947+0000 7ff5e4b48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42548/0 (socket says 172.21.15.64:42548) 2024-05-08T21:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.947+0000 7ff5e4b48700 1 -- 172.21.15.64:0/530098932 learned_addr learned my addr 172.21.15.64:0/530098932 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.949+0000 7ff5dffff700 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.949+0000 7ff5dffff700 1 -- 172.21.15.64:0/530098932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 msgr2=0x5623e7073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.949+0000 7ff5dffff700 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7073ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.949+0000 7ff5dffff700 1 -- 172.21.15.64:0/530098932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 msgr2=0x5623e7075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.949+0000 7ff5dffff700 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e7075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.949+0000 7ff5dffff700 1 -- 172.21.15.64:0/530098932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e6e45350 con 0x5623e70743e0 2024-05-08T21:57:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.952+0000 7ff5dffff700 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7074cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00ca5a0 tx=0x7ff5d0099920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667231908c60f699 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.952+0000 7ff5c6ffd700 1 -- 172.21.15.64:0/530098932 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0099ef0 con 0x5623e70743e0 2024-05-08T21:57:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.952+0000 7ff5c6ffd700 1 -- 172.21.15.64:0/530098932 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d009dd20 con 0x5623e70743e0 2024-05-08T21:57:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.952+0000 7ff5c6ffd700 1 -- 172.21.15.64:0/530098932 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00a1ee0 con 0x5623e70743e0 2024-05-08T21:57:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.954+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/530098932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 msgr2=0x5623e7074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.954+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7074cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00ca5a0 tx=0x7ff5d0099920 comp rx=0 tx=0).stop 2024-05-08T21:57:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.956+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/530098932 shutdown_connections 2024-05-08T21:57:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.956+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e7075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.956+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.956+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/530098932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7074cf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.956+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/530098932 >> 172.21.15.64:0/530098932 conn(0x5623e6fd21b0 msgr2=0x5623e7070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.958+0000 7ff5e5349700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff5c80a1120 0x7ff5c80a35e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097a0 tx=0x7ff5d40119e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.958+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/530098932 shutdown_connections 2024-05-08T21:57:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.962+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/530098932 wait complete. 2024-05-08T21:57:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 -- start start 2024-05-08T21:57:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 Processor -- start 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e707d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 0x5623e707de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70752d0 0x5623e7084ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6f60500 con 0x5623e70743e0 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6f601a0 con 0x5623e70752d0 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.963+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6e47640 con 0x5623e7073a80 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5f2f66b80 10 client.0 _ll_register_callbacks cb 0x5623e6f69ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 0x5623e707de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 0x5623e707de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53082/0 (socket says 172.21.15.64:53082) 2024-05-08T21:57:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5dffff700 1 -- 172.21.15.64:0/3191662841 learned_addr learned my addr 172.21.15.64:0/3191662841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e707d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 -- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70752d0 msgr2=0x5623e7084ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70752d0 0x5623e7084ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 -- 172.21.15.64:0/3191662841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 msgr2=0x5623e707de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 0x5623e707de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 -- 172.21.15.64:0/3191662841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e01a5c00 con 0x5623e7073a80 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5dffff700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 0x5623e707de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e707d950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8007b60 tx=0x7ff5d800baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c12e8944a7da410 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5bffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.964+0000 7ff5bffff700 1 -- 172.21.15.64:0/3191662841 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800e040 con 0x5623e7073a80 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.965+0000 7ff5bffff700 1 -- 172.21.15.64:0/3191662841 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff5d8003c00 con 0x5623e7073a80 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.970+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623e6f1d210 con 0x5623e7073a80 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.970+0000 7ff5bffff700 1 -- 172.21.15.64:0/3191662841 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5d8003dc0 con 0x5623e7073a80 2024-05-08T21:57:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.970+0000 7ff5f2f66b80 10 client.25150 fetch_fsmap learned FSMap version 33 2024-05-08T21:57:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.970+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5623e6df1b00 con 0x5623e7073a80 2024-05-08T21:57:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5bffff700 1 -- 172.21.15.64:0/3191662841 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623e6f1d210 con 0x5623e7073a80 2024-05-08T21:57:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5bffff700 1 -- 172.21.15.64:0/3191662841 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7ff5d800b100 con 0x5623e7073a80 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5f2f66b80 10 client.25150 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5f2f66b80 10 client.25150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5f2f66b80 20 client.25150 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e709a7c0 0x5623e709cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5f2f66b80 4 client.25150 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.971+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623e70437b0 con 0x5623e709a7c0 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.972+0000 7ff5dffff700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e709a7c0 0x5623e709cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.972+0000 7ff5bffff700 10 client.25150 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.972+0000 7ff5dffff700 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e709a7c0 0x5623e709cbe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f33a416f221971 server_cookie=f008cb206280213f in_seq=0 out_seq=0 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5bffff700 1 -- 172.21.15.64:0/3191662841 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623e70437b0 con 0x5623e709a7c0 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5bffff700 10 client.25150 handle_command_reply: tid=0 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 client.25150 shutdown 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 msgr2=0x5623e707d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e707d950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8007b60 tx=0x7ff5d800baa0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 shutdown_connections 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70752d0 0x5623e7084ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e709a7c0 0x5623e709cbe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e707d950 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3191662841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70743e0 0x5623e707de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5bffff700 0 client.25150 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 >> 172.21.15.64:0/3191662841 conn(0x5623e6fd21b0 msgr2=0x5623e706e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 shutdown_connections 2024-05-08T21:57:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.973+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3191662841 wait complete. 2024-05-08T21:57:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.974+0000 7ff5f2f66b80 20 client.25150 trim_cache size 0 max 16384 2024-05-08T21:57:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.998+0000 7ff5f2f66b80 1 Processor -- start 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.998+0000 7ff5f2f66b80 1 -- start start 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7096590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7084e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e70736b0 con 0x5623e70752d0 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e706b9b0 con 0x5623e70743e0 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e7097c10 con 0x5623e7073a80 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70756b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53090/0 (socket says 172.21.15.64:53090) 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 -- 172.21.15.64:0/4246950150 learned_addr learned my addr 172.21.15.64:0/4246950150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5dffff700 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7084e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 -- 172.21.15.64:0/4246950150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 msgr2=0x5623e7096590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7096590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 -- 172.21.15.64:0/4246950150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 msgr2=0x5623e7084e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7084e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:03.999+0000 7ff5e5349700 1 -- 172.21.15.64:0/4246950150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e6f8a500 con 0x5623e70752d0 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.000+0000 7ff5e5349700 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70756b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40112c0 tx=0x7ff5d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea87033b61b22467 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.000+0000 7ff5c77fe700 1 -- 172.21.15.64:0/4246950150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4016070 con 0x5623e70752d0 2024-05-08T21:57:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.000+0000 7ff5c77fe700 1 -- 172.21.15.64:0/4246950150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d40112f0 con 0x5623e70752d0 2024-05-08T21:57:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.000+0000 7ff5c77fe700 1 -- 172.21.15.64:0/4246950150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4027ad0 con 0x5623e70752d0 2024-05-08T21:57:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.001+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/4246950150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 msgr2=0x5623e70756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.001+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70756b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40112c0 tx=0x7ff5d4009260 comp rx=0 tx=0).stop 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.001+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/4246950150 shutdown_connections 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.001+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70756b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.001+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7096590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.001+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/4246950150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7084e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.002+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/4246950150 >> 172.21.15.64:0/4246950150 conn(0x5623e6fd21b0 msgr2=0x5623e707b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.002+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/4246950150 shutdown_connections 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.002+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/4246950150 wait complete. 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.002+0000 7ff5f2f66b80 1 -- start start 2024-05-08T21:57:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.002+0000 7ff5f2f66b80 1 Processor -- start 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7082600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e7083f00 con 0x5623e70752d0 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e7084070 con 0x5623e70743e0 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e70841e0 con 0x5623e7073a80 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5f2f66b80 10 client.0 _ll_register_callbacks cb 0x5623e6f69ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5e4b48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7082600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5e4b48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7082600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42574/0 (socket says 172.21.15.64:42574) 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5e4b48700 1 -- 172.21.15.64:0/3037773268 learned_addr learned my addr 172.21.15.64:0/3037773268 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.004+0000 7ff5e5349700 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5e5349700 1 -- 172.21.15.64:0/3037773268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 msgr2=0x5623e7082600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5e5349700 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7082600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5e5349700 1 -- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 msgr2=0x5623e7082b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5e5349700 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5e5349700 1 -- 172.21.15.64:0/3037773268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4011050 con 0x5623e70752d0 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5e5349700 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70839c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4011de0 tx=0x7ff5d4027d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6de183302c9e20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.005+0000 7ff5bffff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.006+0000 7ff5bffff700 1 -- 172.21.15.64:0/3037773268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d402b070 con 0x5623e70752d0 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.006+0000 7ff5bffff700 1 -- 172.21.15.64:0/3037773268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff5d4004960 con 0x5623e70752d0 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.006+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5d009dd20 con 0x5623e70752d0 2024-05-08T21:57:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.006+0000 7ff5bffff700 1 -- 172.21.15.64:0/3037773268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5d4004b20 con 0x5623e70752d0 2024-05-08T21:57:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 10 client.15492 fetch_fsmap learned FSMap version 33 2024-05-08T21:57:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5623e6e49c70 con 0x5623e70752d0 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5bffff700 1 -- 172.21.15.64:0/3037773268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d009dd20 con 0x5623e70752d0 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5bffff700 1 -- 172.21.15.64:0/3037773268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7498+0+0 (secure 0 0 0) 0x7ff5d4010c40 con 0x5623e70752d0 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 20 client.15492 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e70f1460 0x5623e70f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 4 client.15492 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.010+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623e6fbce60 con 0x5623e70f1460 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.011+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e70f1460 0x5623e70f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.011+0000 7ff5e4b48700 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e70f1460 0x5623e70f3880 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4edc19176c779ec8 server_cookie=d7ac448b7ee25ef in_seq=0 out_seq=0 2024-05-08T21:57:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.011+0000 7ff5bffff700 10 client.15492 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5bffff700 1 -- 172.21.15.64:0/3037773268 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623e6fbce60 con 0x5623e70f1460 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5bffff700 10 client.15492 handle_command_reply: tid=0 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 client.15492 shutdown 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 msgr2=0x5623e70839c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70839c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4011de0 tx=0x7ff5d4027d80 comp rx=0 tx=0).stop 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 shutdown_connections 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5623e70f1460 0x5623e70f3880 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623e70752d0 0x5623e70839c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5623e7073a80 0x5623e7082600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 --2- 172.21.15.64:0/3037773268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e70743e0 0x5623e7082b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 >> 172.21.15.64:0/3037773268 conn(0x5623e6fd21b0 msgr2=0x5623e70a1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.031+0000 7ff5bffff700 0 client.15492 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 shutdown_connections 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5f2f66b80 1 -- 172.21.15.64:0/3037773268 wait complete. 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5f2f66b80 20 client.15492 trim_cache size 0 max 16384 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:57:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:57:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:57:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:57:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff5c80a1120 msgr2=0x7ff5c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff5c80a1120 0x7ff5c80a35e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097a0 tx=0x7ff5d40119e0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 msgr2=0x7ff5e019dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e019dc40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000e9a0 tx=0x7ff5d000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 shutdown_connections 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff5c80a1120 0x7ff5c80a35e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e01090e0 0x7ff5e01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e0107550 0x7ff5e019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 --2- 172.21.15.64:0/2004081981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0108740 0x7ff5e019dc40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 >> 172.21.15.64:0/2004081981 conn(0x7ff5e0076510 msgr2=0x7ff5e010bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 shutdown_connections 2024-05-08T21:57:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:04.032+0000 7ff5e6dac700 1 -- 172.21.15.64:0/2004081981 wait complete. 2024-05-08T21:57:05.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:05 smithi064 ceph-mon[42500]: pgmap v333: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-05-08T21:57:05.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:05 smithi160 ceph-mon[46962]: pgmap v333: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-05-08T21:57:05.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:05 smithi186 ceph-mon[46619]: pgmap v333: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-05-08T21:57:06.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 135.0 s, 2224.2 tps, lat 44.991 ms stddev 49.845 2024-05-08T21:57:07.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:07 smithi064 ceph-mon[42500]: pgmap v334: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 955 op/s 2024-05-08T21:57:07.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:07 smithi160 ceph-mon[46962]: pgmap v334: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 955 op/s 2024-05-08T21:57:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:07 smithi186 ceph-mon[46619]: pgmap v334: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 955 op/s 2024-05-08T21:57:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:09 smithi064 ceph-mon[42500]: pgmap v335: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 1.06k op/s 2024-05-08T21:57:09.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:09 smithi160 ceph-mon[46962]: pgmap v335: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 1.06k op/s 2024-05-08T21:57:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:09 smithi186 ceph-mon[46619]: pgmap v335: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 1.06k op/s 2024-05-08T21:57:11.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 140.0 s, 2316.0 tps, lat 40.949 ms stddev 49.500 2024-05-08T21:57:11.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:11 smithi160 ceph-mon[46962]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.14k op/s 2024-05-08T21:57:11.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:11 smithi186 ceph-mon[46619]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.14k op/s 2024-05-08T21:57:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:11 smithi064 ceph-mon[42500]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.14k op/s 2024-05-08T21:57:13.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:13 smithi160 ceph-mon[46962]: pgmap v337: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 1.36k op/s 2024-05-08T21:57:13.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:13 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:57:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:13 smithi186 ceph-mon[46619]: pgmap v337: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 1.36k op/s 2024-05-08T21:57:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:13 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:57:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:13 smithi064 ceph-mon[42500]: pgmap v337: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 1.36k op/s 2024-05-08T21:57:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:13 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:57:15.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:15 smithi160 ceph-mon[46962]: pgmap v338: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 MiB/s wr, 2.00k op/s 2024-05-08T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:15 smithi186 ceph-mon[46619]: pgmap v338: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 MiB/s wr, 2.00k op/s 2024-05-08T21:57:15.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:15 smithi064 ceph-mon[42500]: pgmap v338: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 MiB/s wr, 2.00k op/s 2024-05-08T21:57:16.479 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 145.0 s, 2944.0 tps, lat 35.488 ms stddev 50.011 2024-05-08T21:57:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.211+0000 7f448b86e700 1 -- 172.21.15.64:0/1402441025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102070 msgr2=0x7f44841024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.211+0000 7f448b86e700 1 --2- 172.21.15.64:0/1402441025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102070 0x7f44841024d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f44800098f0 tx=0x7f4480009c00 comp rx=0 tx=0).stop 2024-05-08T21:57:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 -- 172.21.15.64:0/1402441025 shutdown_connections 2024-05-08T21:57:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 --2- 172.21.15.64:0/1402441025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4484102a10 0x7f448410b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 --2- 172.21.15.64:0/1402441025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44840ff0e0 0x7f44840ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 --2- 172.21.15.64:0/1402441025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102070 0x7f44841024d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 -- 172.21.15.64:0/1402441025 >> 172.21.15.64:0/1402441025 conn(0x7f44840faaf0 msgr2=0x7f44840fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 -- 172.21.15.64:0/1402441025 shutdown_connections 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.212+0000 7f448b86e700 1 -- 172.21.15.64:0/1402441025 wait complete. 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 Processor -- start 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 -- start start 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44840ff0e0 0x7f44841991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4484102070 0x7f44841996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 0x7f44841a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44841a0ce0 con 0x7f4484102a10 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44841a0e20 con 0x7f4484102070 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f448b86e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44841a0fb0 con 0x7f44840ff0e0 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f4489e0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 0x7f44841a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f4489e0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 0x7f44841a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36428/0 (socket says 172.21.15.64:36428) 2024-05-08T21:57:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.213+0000 7f4489e0b700 1 -- 172.21.15.64:0/618135315 learned_addr learned my addr 172.21.15.64:0/618135315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4488e09700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4484102070 0x7f44841996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4489e0b700 1 -- 172.21.15.64:0/618135315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44840ff0e0 msgr2=0x7f44841991b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4489e0b700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44840ff0e0 0x7f44841991b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4489e0b700 1 -- 172.21.15.64:0/618135315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4484102070 msgr2=0x7f44841996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4489e0b700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4484102070 0x7f44841996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4489e0b700 1 -- 172.21.15.64:0/618135315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009520 con 0x7f4484102a10 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.214+0000 7f4489e0b700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 0x7f44841a0710 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f447400d980 tx=0x7f447400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.215+0000 7f447a7fc700 1 -- 172.21.15.64:0/618135315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474004a30 con 0x7f4484102a10 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.215+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4484103410 con 0x7f4484102a10 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.215+0000 7f447a7fc700 1 -- 172.21.15.64:0/618135315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44740054e0 con 0x7f4484102a10 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.215+0000 7f447a7fc700 1 -- 172.21.15.64:0/618135315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474009bf0 con 0x7f4484102a10 2024-05-08T21:57:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.215+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484103970 con 0x7f4484102a10 2024-05-08T21:57:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.217+0000 7f447a7fc700 1 -- 172.21.15.64:0/618135315 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4474009d50 con 0x7f4484102a10 2024-05-08T21:57:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.220+0000 7f447a7fc700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f44700a1170 0x7f44700a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.220+0000 7f447a7fc700 1 -- 172.21.15.64:0/618135315 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44740cc430 con 0x7f4484102a10 2024-05-08T21:57:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.220+0000 7f448960a700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f44700a1170 0x7f44700a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.221+0000 7f448960a700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f44700a1170 0x7f44700a3630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4484068080 tx=0x7f447c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.222+0000 7f4497a28b80 1 Processor -- start 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.222+0000 7f4497a28b80 1 -- start start 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfd5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 0x55d34dfd6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4497a28b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34df2c0f0 con 0x55d34dfd4a80 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4497a28b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34df29ab0 con 0x55d34dfd53e0 2024-05-08T21:57:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4497a28b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34df253a0 con 0x55d34dfd62d0 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfd5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfd5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:48004/0 (socket says 172.21.15.64:48004) 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 -- 172.21.15.64:0/2516861098 learned_addr learned my addr 172.21.15.64:0/2516861098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 -- 172.21.15.64:0/2516861098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 msgr2=0x55d34dfd6750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 0x55d34dfd6750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 -- 172.21.15.64:0/2516861098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 msgr2=0x55d34dfd4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.223+0000 7f4488e09700 1 -- 172.21.15.64:0/2516861098 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34dda6350 con 0x55d34dfd53e0 2024-05-08T21:57:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.224+0000 7f4488e09700 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfd5cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44800098c0 tx=0x7f44800058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb56d2f97533c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.224+0000 7f446f7fe700 1 -- 172.21.15.64:0/2516861098 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448001c050 con 0x55d34dfd53e0 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.224+0000 7f446f7fe700 1 -- 172.21.15.64:0/2516861098 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448001d040 con 0x55d34dfd53e0 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.224+0000 7f446f7fe700 1 -- 172.21.15.64:0/2516861098 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000b820 con 0x55d34dfd53e0 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 -- 172.21.15.64:0/2516861098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 msgr2=0x55d34dfd5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfd5cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44800098c0 tx=0x7f44800058e0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 -- 172.21.15.64:0/2516861098 shutdown_connections 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 0x55d34dfd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2516861098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfd5cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.225+0000 7f4497a28b80 1 -- 172.21.15.64:0/2516861098 >> 172.21.15.64:0/2516861098 conn(0x55d34df331b0 msgr2=0x55d34dfd1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.226+0000 7f4497a28b80 1 -- 172.21.15.64:0/2516861098 shutdown_connections 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.226+0000 7f4497a28b80 1 -- 172.21.15.64:0/2516861098 wait complete. 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.226+0000 7f4497a28b80 1 -- start start 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.226+0000 7f4497a28b80 1 Processor -- start 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfefe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 0x55d34dff6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dec1500 con 0x55d34dfd4a80 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dec11a0 con 0x55d34dfd53e0 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dda8640 con 0x55d34dfd62d0 2024-05-08T21:57:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.227+0000 7f4497a28b80 10 client.0 _ll_register_callbacks cb 0x55d34decaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfefe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfefe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:48014/0 (socket says 172.21.15.64:48014) 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 -- 172.21.15.64:0/2783054184 learned_addr learned my addr 172.21.15.64:0/2783054184 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f448960a700 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 -- 172.21.15.64:0/2783054184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 msgr2=0x55d34dff6ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 0x55d34dff6ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 -- 172.21.15.64:0/2783054184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 msgr2=0x55d34dfef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.228+0000 7f4488e09700 1 -- 172.21.15.64:0/2783054184 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448410be90 con 0x55d34dfd53e0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f4488e09700 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfefe80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f448000fb00 tx=0x7f448000b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5964acfc5e5cef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f446cff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f446cff9700 1 -- 172.21.15.64:0/2783054184 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000b760 con 0x55d34dfd53e0 2024-05-08T21:57:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f446cff9700 1 -- 172.21.15.64:0/2783054184 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f448000b8c0 con 0x55d34dfd53e0 2024-05-08T21:57:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d34de7e210 con 0x55d34dfd53e0 2024-05-08T21:57:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f446cff9700 1 -- 172.21.15.64:0/2783054184 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448001c030 con 0x55d34dfd53e0 2024-05-08T21:57:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f4497a28b80 10 client.25226 fetch_fsmap learned FSMap version 34 2024-05-08T21:57:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d34dd52b00 con 0x55d34dfd53e0 2024-05-08T21:57:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.229+0000 7f446cff9700 1 -- 172.21.15.64:0/2783054184 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d34de7e210 con 0x55d34dfd53e0 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f446cff9700 1 -- 172.21.15.64:0/2783054184 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4480015020 con 0x55d34dfd53e0 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f4497a28b80 10 client.25226 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f4497a28b80 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f4497a28b80 20 client.25226 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34dffb250 0x55d34dffd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f4497a28b80 4 client.25226 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.230+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d34dfa47b0 con 0x55d34dffb250 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.231+0000 7f448960a700 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34dffb250 0x55d34dffd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.231+0000 7f446cff9700 10 client.25226 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.231+0000 7f448960a700 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34dffb250 0x55d34dffd670 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c0d58341005b81 server_cookie=e34fd8c911074067 in_seq=0 out_seq=0 2024-05-08T21:57:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f446cff9700 1 -- 172.21.15.64:0/2783054184 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d34dfa47b0 con 0x55d34dffb250 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f446cff9700 10 client.25226 handle_command_reply: tid=0 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 client.25226 shutdown 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 msgr2=0x55d34dfefe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfefe80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f448000fb00 tx=0x7f448000b5c0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 shutdown_connections 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34dffb250 0x55d34dffd670 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd62d0 0x55d34dff6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd4a80 0x55d34dfef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 --2- 172.21.15.64:0/2783054184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd53e0 0x55d34dfefe80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.232+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 >> 172.21.15.64:0/2783054184 conn(0x55d34df331b0 msgr2=0x55d34dfcf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.233+0000 7f446cff9700 0 client.25226 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.233+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 shutdown_connections 2024-05-08T21:57:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.233+0000 7f4497a28b80 1 -- 172.21.15.64:0/2783054184 wait complete. 2024-05-08T21:57:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.233+0000 7f4497a28b80 20 client.25226 trim_cache size 0 max 16384 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.264+0000 7f4497a28b80 1 Processor -- start 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 -- start start 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd53e0 0x55d34dff6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 0x55d34dfee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34de7e210 con 0x55d34dfd62d0 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dfca570 con 0x55d34dfd4a80 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4497a28b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dff8040 con 0x55d34dfd53e0 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4489e0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 0x55d34dfee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4489e0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 0x55d34dfee4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36448/0 (socket says 172.21.15.64:36448) 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f4489e0b700 1 -- 172.21.15.64:0/400613769 learned_addr learned my addr 172.21.15.64:0/400613769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.268+0000 7f448960a700 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.269+0000 7f4489e0b700 1 -- 172.21.15.64:0/400613769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd53e0 msgr2=0x55d34dff6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.269+0000 7f4489e0b700 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd53e0 0x55d34dff6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.269+0000 7f4489e0b700 1 -- 172.21.15.64:0/400613769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd4a80 msgr2=0x55d34dfd4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.269+0000 7f4489e0b700 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.269+0000 7f4489e0b700 1 -- 172.21.15.64:0/400613769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34deeb500 con 0x55d34dfd62d0 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.269+0000 7f4489e0b700 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 0x55d34dfee4f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4474095d30 tx=0x7f44740996d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ebd55dac5d309a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.271+0000 7f446ffff700 1 -- 172.21.15.64:0/400613769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474005650 con 0x55d34dfd62d0 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.271+0000 7f446ffff700 1 -- 172.21.15.64:0/400613769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4474099f30 con 0x55d34dfd62d0 2024-05-08T21:57:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.271+0000 7f446ffff700 1 -- 172.21.15.64:0/400613769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474097cf0 con 0x55d34dfd62d0 2024-05-08T21:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.275+0000 7f4497a28b80 1 -- 172.21.15.64:0/400613769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 msgr2=0x55d34dfee4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.275+0000 7f4497a28b80 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 0x55d34dfee4f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4474095d30 tx=0x7f44740996d0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 -- 172.21.15.64:0/400613769 shutdown_connections 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd4a80 0x55d34dfd4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd62d0 0x55d34dfee4f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 --2- 172.21.15.64:0/400613769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd53e0 0x55d34dff6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 -- 172.21.15.64:0/400613769 >> 172.21.15.64:0/400613769 conn(0x55d34df331b0 msgr2=0x55d34dfcbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 -- 172.21.15.64:0/400613769 shutdown_connections 2024-05-08T21:57:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.285+0000 7f4497a28b80 1 -- 172.21.15.64:0/400613769 wait complete. 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.286+0000 7f4497a28b80 1 -- start start 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.286+0000 7f4497a28b80 1 Processor -- start 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd4a80 0x55d34dff2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd53e0 0x55d34dff2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd62d0 0x55d34dff5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dff5ca0 con 0x55d34dfd53e0 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dff5e10 con 0x55d34dfd62d0 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dff5fd0 con 0x55d34dfd4a80 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4497a28b80 10 client.0 _ll_register_callbacks cb 0x55d34decaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4488e09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd53e0 0x55d34dff2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4488e09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd53e0 0x55d34dff2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36460/0 (socket says 172.21.15.64:36460) 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4488e09700 1 -- 172.21.15.64:0/1003690236 learned_addr learned my addr 172.21.15.64:0/1003690236 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4489e0b700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd62d0 0x55d34dff5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f448960a700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd4a80 0x55d34dff2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4489e0b700 1 -- 172.21.15.64:0/1003690236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd4a80 msgr2=0x55d34dff2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4489e0b700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd4a80 0x55d34dff2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4489e0b700 1 -- 172.21.15.64:0/1003690236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd53e0 msgr2=0x55d34dff2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4489e0b700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd53e0 0x55d34dff2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.287+0000 7f4489e0b700 1 -- 172.21.15.64:0/1003690236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484103970 con 0x55d34dfd62d0 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.288+0000 7f4489e0b700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd62d0 0x55d34dff5720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44740997c0 tx=0x7f44740994a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff08ffad5387890 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.288+0000 7f446cff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:57:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.291+0000 7f446cff9700 1 -- 172.21.15.64:0/1003690236 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740b1070 con 0x55d34dfd62d0 2024-05-08T21:57:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.291+0000 7f446cff9700 1 -- 172.21.15.64:0/1003690236 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4474099830 con 0x55d34dfd62d0 2024-05-08T21:57:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.291+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f448001d040 con 0x55d34dfd62d0 2024-05-08T21:57:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.295+0000 7f446cff9700 1 -- 172.21.15.64:0/1003690236 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4474097cd0 con 0x55d34dfd62d0 2024-05-08T21:57:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.295+0000 7f4497a28b80 10 client.25232 fetch_fsmap learned FSMap version 34 2024-05-08T21:57:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.295+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d34ddaac70 con 0x55d34dfd62d0 2024-05-08T21:57:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f446cff9700 1 -- 172.21.15.64:0/1003690236 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448001d040 con 0x55d34dfd62d0 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f446cff9700 1 -- 172.21.15.64:0/1003690236 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f44740a1820 con 0x55d34dfd62d0 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f4497a28b80 10 client.25232 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f4497a28b80 10 client.25232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f4497a28b80 20 client.25232 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f4497a28b80 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34e062a20 0x55d34e064e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f4497a28b80 4 client.25232 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d34df1de60 con 0x55d34e062a20 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f448960a700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34e062a20 0x55d34e064e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.297+0000 7f446cff9700 10 client.25232 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.298+0000 7f448960a700 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34e062a20 0x55d34e064e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28c9dbd9160559d server_cookie=6bb916bfda268a3e in_seq=0 out_seq=0 2024-05-08T21:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.311+0000 7f446cff9700 1 -- 172.21.15.64:0/1003690236 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d34df1de60 con 0x55d34e062a20 2024-05-08T21:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.311+0000 7f446cff9700 10 client.25232 handle_command_reply: tid=0 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.311+0000 7f4497a28b80 1 client.25232 shutdown 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.311+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd62d0 msgr2=0x55d34dff5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd62d0 0x55d34dff5720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44740997c0 tx=0x7f44740994a0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 shutdown_connections 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34dfd62d0 0x55d34dff5720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d34e062a20 0x55d34e064e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d34dfd4a80 0x55d34dff2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 --2- 172.21.15.64:0/1003690236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d34dfd53e0 0x55d34dff2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 >> 172.21.15.64:0/1003690236 conn(0x55d34df331b0 msgr2=0x55d34e001fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f446cff9700 0 client.25232 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 shutdown_connections 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 1 -- 172.21.15.64:0/1003690236 wait complete. 2024-05-08T21:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f4497a28b80 20 client.25232 trim_cache size 0 max 16384 2024-05-08T21:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f44700a1170 msgr2=0x7f44700a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.312+0000 7f448b86e700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f44700a1170 0x7f44700a3630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4484068080 tx=0x7f447c009250 comp rx=0 tx=0).stop 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.313+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 msgr2=0x7f44841a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.313+0000 7f448b86e700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 0x7f44841a0710 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f447400d980 tx=0x7f447400dc90 comp rx=0 tx=0).stop 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 shutdown_connections 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f44700a1170 0x7f44700a3630 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4484102a10 0x7f44841a0710 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44840ff0e0 0x7f44841991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 --2- 172.21.15.64:0/618135315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4484102070 0x7f44841996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 >> 172.21.15.64:0/618135315 conn(0x7f44840faaf0 msgr2=0x7f44841077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 shutdown_connections 2024-05-08T21:57:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:17.314+0000 7f448b86e700 1 -- 172.21.15.64:0/618135315 wait complete. 2024-05-08T21:57:17.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:17 smithi160 ceph-mon[46962]: pgmap v339: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 MiB/s wr, 2.13k op/s 2024-05-08T21:57:17.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:17 smithi186 ceph-mon[46619]: pgmap v339: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 MiB/s wr, 2.13k op/s 2024-05-08T21:57:17.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:17 smithi064 ceph-mon[42500]: pgmap v339: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 MiB/s wr, 2.13k op/s 2024-05-08T21:57:19.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:19 smithi160 ceph-mon[46962]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.19k op/s 2024-05-08T21:57:19.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:19 smithi186 ceph-mon[46619]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.19k op/s 2024-05-08T21:57:19.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:19 smithi064 ceph-mon[42500]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.19k op/s 2024-05-08T21:57:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 150.0 s, 3178.8 tps, lat 31.690 ms stddev 35.161 2024-05-08T21:57:21.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:21 smithi160 ceph-mon[46962]: pgmap v341: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 MiB/s wr, 2.31k op/s 2024-05-08T21:57:21.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:21 smithi186 ceph-mon[46619]: pgmap v341: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 MiB/s wr, 2.31k op/s 2024-05-08T21:57:21.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:21 smithi064 ceph-mon[42500]: pgmap v341: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 MiB/s wr, 2.31k op/s 2024-05-08T21:57:24.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:23 smithi160 ceph-mon[46962]: pgmap v342: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 48 MiB/s wr, 2.28k op/s 2024-05-08T21:57:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:23 smithi186 ceph-mon[46619]: pgmap v342: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 48 MiB/s wr, 2.28k op/s 2024-05-08T21:57:24.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:23 smithi064 ceph-mon[42500]: pgmap v342: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 48 MiB/s wr, 2.28k op/s 2024-05-08T21:57:26.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:26 smithi160 ceph-mon[46962]: pgmap v343: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 MiB/s wr, 2.04k op/s 2024-05-08T21:57:26.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:26 smithi186 ceph-mon[46619]: pgmap v343: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 MiB/s wr, 2.04k op/s 2024-05-08T21:57:26.462 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:26 smithi064 ceph-mon[42500]: pgmap v343: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 MiB/s wr, 2.04k op/s 2024-05-08T21:57:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 155.0 s, 3340.6 tps, lat 29.706 ms stddev 31.466 2024-05-08T21:57:27.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:27 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:57:27.399 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:27 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:57:27.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:27 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:57:28.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:28 smithi186 ceph-mon[46619]: pgmap v344: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 1.31k op/s 2024-05-08T21:57:28.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:57:28.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:57:28.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:57:28.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:28 smithi064 ceph-mon[42500]: pgmap v344: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 1.31k op/s 2024-05-08T21:57:28.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:57:28.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:57:28.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:57:28.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:28 smithi160 ceph-mon[46962]: pgmap v344: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 1.31k op/s 2024-05-08T21:57:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:57:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:57:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:57:29.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:29 smithi186 ceph-mon[46619]: pgmap v345: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-05-08T21:57:29.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:29 smithi064 ceph-mon[42500]: pgmap v345: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-05-08T21:57:29.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:29 smithi160 ceph-mon[46962]: pgmap v345: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.499+0000 7f7d237a0700 1 -- 172.21.15.64:0/3176186649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c10a910 msgr2=0x7f7d1c10cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.499+0000 7f7d237a0700 1 --2- 172.21.15.64:0/3176186649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c10a910 0x7f7d1c10cda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009990 tx=0x7f7d0c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.500+0000 7f7d237a0700 1 -- 172.21.15.64:0/3176186649 shutdown_connections 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.500+0000 7f7d237a0700 1 --2- 172.21.15.64:0/3176186649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.500+0000 7f7d237a0700 1 --2- 172.21.15.64:0/3176186649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d1c107fa0 0x7f7d1c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.500+0000 7f7d237a0700 1 --2- 172.21.15.64:0/3176186649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c10a910 0x7f7d1c10cda0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009990 tx=0x7f7d0c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.500+0000 7f7d237a0700 1 -- 172.21.15.64:0/3176186649 >> 172.21.15.64:0/3176186649 conn(0x7f7d1c06df30 msgr2=0x7f7d1c070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.504+0000 7f7d237a0700 1 -- 172.21.15.64:0/3176186649 shutdown_connections 2024-05-08T21:57:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.504+0000 7f7d237a0700 1 -- 172.21.15.64:0/3176186649 wait complete. 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.505+0000 7f7d237a0700 1 Processor -- start 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.505+0000 7f7d237a0700 1 -- start start 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d237a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d1c107fa0 0x7f7d1c1a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d237a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c1a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d237a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c1ab4e0 0x7f7d1c1ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d237a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1adea0 con 0x7f7d1c107fa0 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d237a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1adfe0 con 0x7f7d1c1ab4e0 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d237a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1ae170 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c1a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c1a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48862/0 (socket says 172.21.15.64:48862) 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 -- 172.21.15.64:0/2784302250 learned_addr learned my addr 172.21.15.64:0/2784302250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c1ab4e0 0x7f7d1c1ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d2153c700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d1c107fa0 0x7f7d1c1a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 -- 172.21.15.64:0/2784302250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c1ab4e0 msgr2=0x7f7d1c1ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c1ab4e0 0x7f7d1c1ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 -- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d1c107fa0 msgr2=0x7f7d1c1a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d1c107fa0 0x7f7d1c1a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.506+0000 7f7d20d3b700 1 -- 172.21.15.64:0/2784302250 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0095f0 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.507+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c1a64d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0098e0 tx=0x7f7d0c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.507+0000 7f7d127fc700 1 -- 172.21.15.64:0/2784302250 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c01d070 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.507+0000 7f7d127fc700 1 -- 172.21.15.64:0/2784302250 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d0c005e10 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.507+0000 7f7d127fc700 1 -- 172.21.15.64:0/2784302250 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005310 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.508+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c1ae3f0 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.508+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c1ae760 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.513+0000 7f7d2f95ab80 1 Processor -- start 2024-05-08T21:57:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.513+0000 7f7d127fc700 1 -- 172.21.15.64:0/2784302250 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7d0c0049c0 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d127fc700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d080a1220 0x7f7d080a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 -- start start 2024-05-08T21:57:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d127fc700 1 -- 172.21.15.64:0/2784302250 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d0c0cd4f0 con 0x7f7d1c10d2e0 2024-05-08T21:57:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 0x563ef40a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40aa2d0 0x563ef40aa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40000f0 con 0x563ef40a93e0 2024-05-08T21:57:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef3ffdab0 con 0x563ef40a8a80 2024-05-08T21:57:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.514+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef3ff93a0 con 0x563ef40aa2d0 2024-05-08T21:57:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 0x563ef40a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 0x563ef40a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47814/0 (socket says 172.21.15.64:47814) 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 -- 172.21.15.64:0/3517556560 learned_addr learned my addr 172.21.15.64:0/3517556560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 -- 172.21.15.64:0/3517556560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40aa2d0 msgr2=0x563ef40aa750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40aa2d0 0x563ef40aa750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 -- 172.21.15.64:0/3517556560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 msgr2=0x563ef40a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 -- 172.21.15.64:0/3517556560 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef3e7a350 con 0x563ef40a8a80 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d080a1220 0x7f7d080a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d2153c700 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 0x563ef40a8ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009af0 tx=0x7f7d18009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54377b60bc61771 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d077fe700 1 -- 172.21.15.64:0/3517556560 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180095f0 con 0x563ef40a8a80 2024-05-08T21:57:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d077fe700 1 -- 172.21.15.64:0/3517556560 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1800ee50 con 0x563ef40a8a80 2024-05-08T21:57:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.515+0000 7f7d077fe700 1 -- 172.21.15.64:0/3517556560 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18012630 con 0x563ef40a8a80 2024-05-08T21:57:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.516+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/3517556560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 msgr2=0x563ef40a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.516+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 0x563ef40a8ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009af0 tx=0x7f7d18009e00 comp rx=0 tx=0).stop 2024-05-08T21:57:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.516+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d080a1220 0x7f7d080a36e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563ef40a9ab0 tx=0x7f7d1400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/3517556560 shutdown_connections 2024-05-08T21:57:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a8a80 0x563ef40a8ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40aa2d0 0x563ef40aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/3517556560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/3517556560 >> 172.21.15.64:0/3517556560 conn(0x563ef40071b0 msgr2=0x563ef40a5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/3517556560 shutdown_connections 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/3517556560 wait complete. 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.517+0000 7f7d2f95ab80 1 -- start start 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 Processor -- start 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a8a80 0x563ef40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a93e0 0x563ef40b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef3f95500 con 0x563ef40a8a80 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef3f951a0 con 0x563ef40aa2d0 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef3e7c640 con 0x563ef40a93e0 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.518+0000 7f7d2f95ab80 10 client.0 _ll_register_callbacks cb 0x563ef3f9ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d21d3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d2153c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a8a80 0x563ef40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d2153c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a8a80 0x563ef40b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47786/0 (socket says 172.21.15.64:47786) 2024-05-08T21:57:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d2153c700 1 -- 172.21.15.64:0/1125223225 learned_addr learned my addr 172.21.15.64:0/1125223225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d21d3d700 1 -- 172.21.15.64:0/1125223225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a93e0 msgr2=0x563ef40b2e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:57:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a93e0 0x563ef40b2e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d21d3d700 1 -- 172.21.15.64:0/1125223225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a8a80 msgr2=0x563ef40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a8a80 0x563ef40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.519+0000 7f7d21d3d700 1 -- 172.21.15.64:0/1125223225 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d18009750 con 0x563ef40aa2d0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.520+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40b9e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7d14016de0 tx=0x7f7d14004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97de4ffea9350409 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.520+0000 7f7d04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.520+0000 7f7d04ff9700 1 -- 172.21.15.64:0/1125223225 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14010040 con 0x563ef40aa2d0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.520+0000 7f7d04ff9700 1 -- 172.21.15.64:0/1125223225 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d14006e40 con 0x563ef40aa2d0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.520+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ef3f52210 con 0x563ef40aa2d0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.521+0000 7f7d04ff9700 1 -- 172.21.15.64:0/1125223225 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d14015d10 con 0x563ef40aa2d0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.521+0000 7f7d2f95ab80 10 client.25244 fetch_fsmap learned FSMap version 34 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.521+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563ef3e26b00 con 0x563ef40aa2d0 2024-05-08T21:57:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.521+0000 7f7d04ff9700 1 -- 172.21.15.64:0/1125223225 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ef3f52210 con 0x563ef40aa2d0 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.521+0000 7f7d04ff9700 1 -- 172.21.15.64:0/1125223225 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7d1401f070 con 0x563ef40aa2d0 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2f95ab80 10 client.25244 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2f95ab80 10 client.25244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2f95ab80 20 client.25244 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef40be890 0x563ef40c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2f95ab80 4 client.25244 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ef40787b0 con 0x563ef40be890 2024-05-08T21:57:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.522+0000 7f7d2153c700 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef40be890 0x563ef40c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.523+0000 7f7d2153c700 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef40be890 0x563ef40c0cb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3e1a96459289ee server_cookie=719dde03a03fb44c in_seq=0 out_seq=0 2024-05-08T21:57:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.523+0000 7f7d04ff9700 10 client.25244 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.523+0000 7f7d04ff9700 1 -- 172.21.15.64:0/1125223225 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ef40787b0 con 0x563ef40be890 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.523+0000 7f7d04ff9700 10 client.25244 handle_command_reply: tid=0 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.523+0000 7f7d2f95ab80 1 client.25244 shutdown 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 msgr2=0x563ef40b9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40b9e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7d14016de0 tx=0x7f7d14004be0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 shutdown_connections 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40b9e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef40be890 0x563ef40c0cb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a8a80 0x563ef40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/1125223225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a93e0 0x563ef40b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 >> 172.21.15.64:0/1125223225 conn(0x563ef40071b0 msgr2=0x563ef40a3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d04ff9700 0 client.25244 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 shutdown_connections 2024-05-08T21:57:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/1125223225 wait complete. 2024-05-08T21:57:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.524+0000 7f7d2f95ab80 20 client.25244 trim_cache size 0 max 16384 2024-05-08T21:57:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 Processor -- start 2024-05-08T21:57:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 -- start start 2024-05-08T21:57:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40b1390 con 0x563ef40a93e0 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40bafc0 con 0x563ef40aa2d0 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.545+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40b2430 con 0x563ef40a8a80 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47830/0 (socket says 172.21.15.64:47830) 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 -- 172.21.15.64:0/2053528501 learned_addr learned my addr 172.21.15.64:0/2053528501 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d2153c700 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 -- 172.21.15.64:0/2053528501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 msgr2=0x563ef40ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 -- 172.21.15.64:0/2053528501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 msgr2=0x563ef40b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.546+0000 7f7d21d3d700 1 -- 172.21.15.64:0/2053528501 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef3fbf500 con 0x563ef40aa2d0 2024-05-08T21:57:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.547+0000 7f7d21d3d700 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40aa6b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009af0 tx=0x7f7d14000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ca92f071bf93031 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.547+0000 7f7d07fff700 1 -- 172.21.15.64:0/2053528501 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14010050 con 0x563ef40aa2d0 2024-05-08T21:57:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.548+0000 7f7d07fff700 1 -- 172.21.15.64:0/2053528501 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1401c040 con 0x563ef40aa2d0 2024-05-08T21:57:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.548+0000 7f7d07fff700 1 -- 172.21.15.64:0/2053528501 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14029d40 con 0x563ef40aa2d0 2024-05-08T21:57:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.549+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/2053528501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 msgr2=0x563ef40aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.549+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40aa6b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009af0 tx=0x7f7d14000f40 comp rx=0 tx=0).stop 2024-05-08T21:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.550+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/2053528501 shutdown_connections 2024-05-08T21:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.550+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40aa2d0 0x563ef40aa6b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.550+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.550+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/2053528501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40a93e0 0x563ef40b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.550+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/2053528501 >> 172.21.15.64:0/2053528501 conn(0x563ef40071b0 msgr2=0x563ef40b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.551+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/2053528501 shutdown_connections 2024-05-08T21:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.551+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/2053528501 wait complete. 2024-05-08T21:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.552+0000 7f7d2f95ab80 1 -- start start 2024-05-08T21:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.552+0000 7f7d2f95ab80 1 Processor -- start 2024-05-08T21:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a93e0 0x563ef40b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40aa2d0 0x563ef40b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40b9130 con 0x563ef40aa2d0 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40b92a0 con 0x563ef40a93e0 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef40b9410 con 0x563ef40a8a80 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2f95ab80 10 client.0 _ll_register_callbacks cb 0x563ef3f9ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2153c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2153c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40b78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48894/0 (socket says 172.21.15.64:48894) 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.553+0000 7f7d2153c700 1 -- 172.21.15.64:0/742065895 learned_addr learned my addr 172.21.15.64:0/742065895 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a93e0 0x563ef40b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d2153c700 1 -- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a93e0 msgr2=0x563ef40b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d2153c700 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a93e0 0x563ef40b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d2153c700 1 -- 172.21.15.64:0/742065895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40aa2d0 msgr2=0x563ef40b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d2153c700 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40aa2d0 0x563ef40b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d2153c700 1 -- 172.21.15.64:0/742065895 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ef3e26b00 con 0x563ef40a8a80 2024-05-08T21:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d2153c700 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40b78d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18013910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ec5c7a3da679a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d04ff9700 1 -- 172.21.15.64:0/742065895 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18021040 con 0x563ef40a8a80 2024-05-08T21:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.554+0000 7f7d04ff9700 1 -- 172.21.15.64:0/742065895 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d1800eb90 con 0x563ef40a8a80 2024-05-08T21:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d1800ee50 con 0x563ef40a8a80 2024-05-08T21:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d04ff9700 1 -- 172.21.15.64:0/742065895 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d18005a50 con 0x563ef40a8a80 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 10 client.25165 fetch_fsmap learned FSMap version 34 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563ef3e7ec70 con 0x563ef40a8a80 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d04ff9700 1 -- 172.21.15.64:0/742065895 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1800ee50 con 0x563ef40a8a80 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d04ff9700 1 -- 172.21.15.64:0/742065895 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7d18011070 con 0x563ef40a8a80 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 10 client.25165 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 10 client.25165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 20 client.25165 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef41267b0 0x563ef4128bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 4 client.25165 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:57:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.555+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563ef3ff1e60 con 0x563ef41267b0 2024-05-08T21:57:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.556+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef41267b0 0x563ef4128bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.557+0000 7f7d20d3b700 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef41267b0 0x563ef4128bd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373cf27227af688f server_cookie=2c20aab9f8cf3854 in_seq=0 out_seq=0 2024-05-08T21:57:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.557+0000 7f7d04ff9700 10 client.25165 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d04ff9700 1 -- 172.21.15.64:0/742065895 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563ef3ff1e60 con 0x563ef41267b0 2024-05-08T21:57:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d04ff9700 10 client.25165 handle_command_reply: tid=0 2024-05-08T21:57:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 client.25165 shutdown 2024-05-08T21:57:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 msgr2=0x563ef40b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40b78d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18013910 comp rx=0 tx=0).stop 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 shutdown_connections 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x563ef41267b0 0x563ef4128bd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ef40aa2d0 0x563ef40b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ef40a8a80 0x563ef40b78d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 --2- 172.21.15.64:0/742065895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ef40a93e0 0x563ef40b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 >> 172.21.15.64:0/742065895 conn(0x563ef40071b0 msgr2=0x563ef40b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d04ff9700 0 client.25165 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 shutdown_connections 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 1 -- 172.21.15.64:0/742065895 wait complete. 2024-05-08T21:57:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.573+0000 7f7d2f95ab80 20 client.25165 trim_cache size 0 max 16384 2024-05-08T21:57:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:57:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:57:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:57:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d080a1220 msgr2=0x7f7d080a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d080a1220 0x7f7d080a36e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563ef40a9ab0 tx=0x7f7d1400d040 comp rx=0 tx=0).stop 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 msgr2=0x7f7d1c1a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c1a64d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0098e0 tx=0x7f7d0c00f690 comp rx=0 tx=0).stop 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 shutdown_connections 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d080a1220 0x7f7d080a36e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d1c1ab4e0 0x7f7d1c1ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d1c107fa0 0x7f7d1c1a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 --2- 172.21.15.64:0/2784302250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c10d2e0 0x7f7d1c1a64d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 >> 172.21.15.64:0/2784302250 conn(0x7f7d1c06df30 msgr2=0x7f7d1c1083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.575+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 shutdown_connections 2024-05-08T21:57:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:30.576+0000 7f7d237a0700 1 -- 172.21.15.64:0/2784302250 wait complete. 2024-05-08T21:57:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 160.0 s, 1869.8 tps, lat 52.754 ms stddev 61.526 2024-05-08T21:57:31.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:31 smithi160 ceph-mon[46962]: pgmap v346: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-05-08T21:57:31.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:31 smithi186 ceph-mon[46619]: pgmap v346: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-05-08T21:57:31.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:31 smithi064 ceph-mon[42500]: pgmap v346: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-05-08T21:57:33.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:33 smithi160 ceph-mon[46962]: pgmap v347: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.05k op/s 2024-05-08T21:57:33.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:33 smithi186 ceph-mon[46619]: pgmap v347: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.05k op/s 2024-05-08T21:57:33.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:33 smithi064 ceph-mon[42500]: pgmap v347: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.05k op/s 2024-05-08T21:57:35.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:35 smithi160 ceph-mon[46962]: pgmap v348: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.11k op/s 2024-05-08T21:57:35.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:35 smithi186 ceph-mon[46619]: pgmap v348: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.11k op/s 2024-05-08T21:57:35.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:35 smithi064 ceph-mon[42500]: pgmap v348: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.11k op/s 2024-05-08T21:57:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 165.0 s, 1964.8 tps, lat 46.619 ms stddev 55.730 2024-05-08T21:57:37.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:37 smithi160 ceph-mon[46962]: pgmap v349: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.13k op/s 2024-05-08T21:57:37.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:37 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:57:37.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:37 smithi186 ceph-mon[46619]: pgmap v349: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.13k op/s 2024-05-08T21:57:37.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:37 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:57:37.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:37 smithi064 ceph-mon[42500]: pgmap v349: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.13k op/s 2024-05-08T21:57:37.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:37 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:57:39.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:39 smithi160 ceph-mon[46962]: pgmap v350: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 MiB/s wr, 1.62k op/s 2024-05-08T21:57:39.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:39 smithi186 ceph-mon[46619]: pgmap v350: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 MiB/s wr, 1.62k op/s 2024-05-08T21:57:39.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:39 smithi064 ceph-mon[42500]: pgmap v350: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 MiB/s wr, 1.62k op/s 2024-05-08T21:57:41.478 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 170.0 s, 2532.9 tps, lat 43.388 ms stddev 74.408 2024-05-08T21:57:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:41 smithi186 ceph-mon[46619]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 MiB/s wr, 2.30k op/s 2024-05-08T21:57:41.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:41 smithi064 ceph-mon[42500]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 MiB/s wr, 2.30k op/s 2024-05-08T21:57:41.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:41 smithi160 ceph-mon[46962]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 MiB/s wr, 2.30k op/s 2024-05-08T21:57:43.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:43 smithi186 ceph-mon[46619]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.20k op/s 2024-05-08T21:57:43.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:43 smithi064 ceph-mon[42500]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.20k op/s 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2959155071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 msgr2=0x7f99a40ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2959155071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a40ff9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009990 tx=0x7f99a0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2959155071 shutdown_connections 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2959155071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99a4101110 0x7f99a41079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2959155071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a40ff9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2959155071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4100770 0x7f99a4100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2959155071 >> 172.21.15.64:0/2959155071 conn(0x7f99a40faaf0 msgr2=0x7f99a40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2959155071 shutdown_connections 2024-05-08T21:57:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.746+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2959155071 wait complete. 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 Processor -- start 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 -- start start 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a410cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99a4100770 0x7f99a410b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4101110 0x7f99a410b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419ae00 con 0x7f99a4101110 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a410bd70 con 0x7f99a4100770 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.747+0000 7f99ac3f7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a410bf30 con 0x7f99a40ff580 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a410cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a410cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56300/0 (socket says 172.21.15.64:56300) 2024-05-08T21:57:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 -- 172.21.15.64:0/2774480604 learned_addr learned my addr 172.21.15.64:0/2774480604 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 -- 172.21.15.64:0/2774480604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99a4100770 msgr2=0x7f99a410b2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:57:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99a4100770 0x7f99a410b2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 -- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4101110 msgr2=0x7f99a410b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4101110 0x7f99a410b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.748+0000 7f99aa193700 1 -- 172.21.15.64:0/2774480604 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00095f0 con 0x7f99a40ff580 2024-05-08T21:57:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.749+0000 7f99aa193700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a410cc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99a0006010 tx=0x7f99a000f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.749+0000 7f999b7fe700 1 -- 172.21.15.64:0/2774480604 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0003c30 con 0x7f99a40ff580 2024-05-08T21:57:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.749+0000 7f999b7fe700 1 -- 172.21.15.64:0/2774480604 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a0003d90 con 0x7f99a40ff580 2024-05-08T21:57:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.749+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a410c150 con 0x7f99a40ff580 2024-05-08T21:57:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.749+0000 7f999b7fe700 1 -- 172.21.15.64:0/2774480604 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a000fe70 con 0x7f99a40ff580 2024-05-08T21:57:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.749+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41a59e0 con 0x7f99a40ff580 2024-05-08T21:57:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.757+0000 7f99b85b1b80 1 Processor -- start 2024-05-08T21:57:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.757+0000 7f999b7fe700 1 -- 172.21.15.64:0/2774480604 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f99a0029460 con 0x7f99a40ff580 2024-05-08T21:57:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.757+0000 7f99b85b1b80 1 -- start start 2024-05-08T21:57:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.758+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.758+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b03e0 0x560f4c2b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.758+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b12d0 0x560f4c2b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.758+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2070f0 con 0x560f4c2b03e0 2024-05-08T21:57:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.758+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c204ab0 con 0x560f4c2afa80 2024-05-08T21:57:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.758+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2003a0 con 0x560f4c2b12d0 2024-05-08T21:57:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60566/0 (socket says 172.21.15.64:60566) 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 -- 172.21.15.64:0/3316791726 learned_addr learned my addr 172.21.15.64:0/3316791726 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa193700 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b03e0 0x560f4c2b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 -- 172.21.15.64:0/3316791726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b12d0 msgr2=0x560f4c2b1750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b12d0 0x560f4c2b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 -- 172.21.15.64:0/3316791726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b03e0 msgr2=0x560f4c2b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b03e0 0x560f4c2b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.759+0000 7f99aa994700 1 -- 172.21.15.64:0/3316791726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f4c081350 con 0x560f4c2afa80 2024-05-08T21:57:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.761+0000 7f999b7fe700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f99900a1270 0x7f99900a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.761+0000 7f99aa193700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f99900a1270 0x7f99900a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.762+0000 7f999b7fe700 1 -- 172.21.15.64:0/2774480604 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99a0012070 con 0x7f99a40ff580 2024-05-08T21:57:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.762+0000 7f99aa193700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f99900a1270 0x7f99900a3730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f99a00095c0 tx=0x7f99a00d16f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.762+0000 7f99aa994700 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2afea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f999c009d00 tx=0x7f999c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86223af0ca107026 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.763+0000 7f9998da4700 1 -- 172.21.15.64:0/3316791726 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00e070 con 0x560f4c2afa80 2024-05-08T21:57:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.763+0000 7f9998da4700 1 -- 172.21.15.64:0/3316791726 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999c009830 con 0x560f4c2afa80 2024-05-08T21:57:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.763+0000 7f9998da4700 1 -- 172.21.15.64:0/3316791726 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c011390 con 0x560f4c2afa80 2024-05-08T21:57:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3316791726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 msgr2=0x560f4c2afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2afea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f999c009d00 tx=0x7f999c00b330 comp rx=0 tx=0).stop 2024-05-08T21:57:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3316791726 shutdown_connections 2024-05-08T21:57:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2afea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b12d0 0x560f4c2b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3316791726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b03e0 0x560f4c2b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.764+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3316791726 >> 172.21.15.64:0/3316791726 conn(0x560f4c20e1b0 msgr2=0x560f4c2ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.765+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3316791726 shutdown_connections 2024-05-08T21:57:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.765+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3316791726 wait complete. 2024-05-08T21:57:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.765+0000 7f99b85b1b80 1 -- start start 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.765+0000 7f99b85b1b80 1 Processor -- start 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 0x560f4c2cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c19c500 con 0x560f4c2b12d0 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c19c1a0 con 0x560f4c2afa80 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c083640 con 0x560f4c2b03e0 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99b85b1b80 10 client.0 _ll_register_callbacks cb 0x560f4c1a5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99aa994700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99aa994700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99a9992700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 0x560f4c2cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99aa994700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37148/0 (socket says 172.21.15.64:37148) 2024-05-08T21:57:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99a9992700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 0x560f4c2cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56312/0 (socket says 172.21.15.64:56312) 2024-05-08T21:57:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.766+0000 7f99aa994700 1 -- 172.21.15.64:0/3024268418 learned_addr learned my addr 172.21.15.64:0/3024268418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99aa994700 1 -- 172.21.15.64:0/3024268418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 msgr2=0x560f4c2cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99aa994700 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 0x560f4c2cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99aa994700 1 -- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 msgr2=0x560f4c2ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99aa994700 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99aa994700 1 -- 172.21.15.64:0/3024268418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c0099b0 con 0x560f4c2b12d0 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99a9992700 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 0x560f4c2cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99aa994700 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d1e50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f999c009f90 tx=0x7f999c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbca93857228eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f998dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f998dffb700 1 -- 172.21.15.64:0/3024268418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00e040 con 0x560f4c2b12d0 2024-05-08T21:57:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f998dffb700 1 -- 172.21.15.64:0/3024268418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f999c011cb0 con 0x560f4c2b12d0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f4c27f7b0 con 0x560f4c2b12d0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f998dffb700 1 -- 172.21.15.64:0/3024268418 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c011390 con 0x560f4c2b12d0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99b85b1b80 10 client.15528 fetch_fsmap learned FSMap version 35 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560f4c02db00 con 0x560f4c2b12d0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.767+0000 7f998dffb700 1 -- 172.21.15.64:0/3024268418 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f4c27f7b0 con 0x560f4c2b12d0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f998dffb700 1 -- 172.21.15.64:0/3024268418 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f999c01e420 con 0x560f4c2b12d0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f99b85b1b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f99b85b1b80 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f99b85b1b80 20 client.15528 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2d66b0 0x560f4c2d8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f99b85b1b80 4 client.15528 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.768+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f4c1f8e60 con 0x560f4c2d66b0 2024-05-08T21:57:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.769+0000 7f99a9992700 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2d66b0 0x560f4c2d8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.769+0000 7f998dffb700 10 client.15528 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.769+0000 7f99a9992700 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2d66b0 0x560f4c2d8ad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977b653eee7ce823 server_cookie=a76b2ed53df5417d in_seq=0 out_seq=0 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f998dffb700 1 -- 172.21.15.64:0/3024268418 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f4c1f8e60 con 0x560f4c2d66b0 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f998dffb700 10 client.15528 handle_command_reply: tid=0 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 client.15528 shutdown 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 msgr2=0x560f4c2d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d1e50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f999c009f90 tx=0x7f999c000f40 comp rx=0 tx=0).stop 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 shutdown_connections 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2afa80 0x560f4c2ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2d66b0 0x560f4c2d8ad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d1e50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.771+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/3024268418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2b03e0 0x560f4c2cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.772+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 >> 172.21.15.64:0/3024268418 conn(0x560f4c20e1b0 msgr2=0x560f4c2aa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.772+0000 7f998dffb700 0 client.15528 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.772+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 shutdown_connections 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.772+0000 7f99b85b1b80 1 -- 172.21.15.64:0/3024268418 wait complete. 2024-05-08T21:57:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.773+0000 7f99b85b1b80 20 client.15528 trim_cache size 0 max 16384 2024-05-08T21:57:43.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:43 smithi160 ceph-mon[46962]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.20k op/s 2024-05-08T21:57:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 Processor -- start 2024-05-08T21:57:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 -- start start 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2a5360 con 0x560f4c2b12d0 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2b1180 con 0x560f4c2b03e0 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2cbac0 con 0x560f4c2afa80 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99a9992700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.796+0000 7f99a9992700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2d1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60572/0 (socket says 172.21.15.64:60572) 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 -- 172.21.15.64:0/258422726 learned_addr learned my addr 172.21.15.64:0/258422726 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 -- 172.21.15.64:0/258422726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 msgr2=0x560f4c2ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 -- 172.21.15.64:0/258422726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 msgr2=0x560f4c2cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 -- 172.21.15.64:0/258422726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f4c1c6500 con 0x560f4c2b03e0 2024-05-08T21:57:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.797+0000 7f99a9992700 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2d1e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9994019d10 tx=0x7f999400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755045177c90bfdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99995a5700 1 -- 172.21.15.64:0/258422726 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994016040 con 0x560f4c2b03e0 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99995a5700 1 -- 172.21.15.64:0/258422726 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999401d070 con 0x560f4c2b03e0 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99995a5700 1 -- 172.21.15.64:0/258422726 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994004a90 con 0x560f4c2b03e0 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99b85b1b80 1 -- 172.21.15.64:0/258422726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 msgr2=0x560f4c2d1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2d1e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9994019d10 tx=0x7f999400b260 comp rx=0 tx=0).stop 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99b85b1b80 1 -- 172.21.15.64:0/258422726 shutdown_connections 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.798+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/258422726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2d1e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 -- 172.21.15.64:0/258422726 >> 172.21.15.64:0/258422726 conn(0x560f4c20e1b0 msgr2=0x560f4c2b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 -- 172.21.15.64:0/258422726 shutdown_connections 2024-05-08T21:57:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 -- 172.21.15.64:0/258422726 wait complete. 2024-05-08T21:57:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 -- start start 2024-05-08T21:57:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 Processor -- start 2024-05-08T21:57:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.799+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99b85b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2d0f40 con 0x560f4c2b12d0 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2d10b0 con 0x560f4c2b03e0 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99b85b1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4c2d1270 con 0x560f4c2afa80 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99b85b1b80 10 client.0 _ll_register_callbacks cb 0x560f4c1a5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37162/0 (socket says 172.21.15.64:37162) 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 -- 172.21.15.64:0/2976720441 learned_addr learned my addr 172.21.15.64:0/2976720441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 -- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 msgr2=0x560f4c2cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 -- 172.21.15.64:0/2976720441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 msgr2=0x560f4c2d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.800+0000 7f99aa994700 1 -- 172.21.15.64:0/2976720441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999400b740 con 0x560f4c2afa80 2024-05-08T21:57:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.801+0000 7f99aa994700 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2cf690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f999c002230 tx=0x7f999c011600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6b756ffe29a8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.801+0000 7f998dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:57:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.801+0000 7f998dffb700 1 -- 172.21.15.64:0/2976720441 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00e050 con 0x560f4c2afa80 2024-05-08T21:57:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.801+0000 7f998dffb700 1 -- 172.21.15.64:0/2976720441 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f999c0126d0 con 0x560f4c2afa80 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.802+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f4c159210 con 0x560f4c2afa80 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f998dffb700 1 -- 172.21.15.64:0/2976720441 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c011cc0 con 0x560f4c2afa80 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 10 client.25177 fetch_fsmap learned FSMap version 35 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560f4c085c70 con 0x560f4c2afa80 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f998dffb700 1 -- 172.21.15.64:0/2976720441 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f4c159210 con 0x560f4c2afa80 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f998dffb700 1 -- 172.21.15.64:0/2976720441 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f999c019410 con 0x560f4c2afa80 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 10 client.25177 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T21:57:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 10 client.25177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 20 client.25177 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2b8520 0x560f4c2ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 4 client.25177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f4c04ec90 con 0x560f4c2b8520 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.803+0000 7f99a9992700 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2b8520 0x560f4c2ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.804+0000 7f99a9992700 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2b8520 0x560f4c2ba940 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db29608f867604f server_cookie=57a5e12038281566 in_seq=0 out_seq=0 2024-05-08T21:57:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.804+0000 7f998dffb700 10 client.25177 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:57:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:57:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:57:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:57:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.817+0000 7f998dffb700 1 -- 172.21.15.64:0/2976720441 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f4c04ec90 con 0x560f4c2b8520 2024-05-08T21:57:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.817+0000 7f998dffb700 10 client.25177 handle_command_reply: tid=0 2024-05-08T21:57:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.817+0000 7f99b85b1b80 1 client.25177 shutdown 2024-05-08T21:57:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 msgr2=0x560f4c2cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2cf690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f999c002230 tx=0x7f999c011600 comp rx=0 tx=0).stop 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 shutdown_connections 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560f4c2b8520 0x560f4c2ba940 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f4c2b12d0 0x560f4c2d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f4c2afa80 0x560f4c2cf690 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 --2- 172.21.15.64:0/2976720441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560f4c2b03e0 0x560f4c2cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 >> 172.21.15.64:0/2976720441 conn(0x560f4c20e1b0 msgr2=0x560f4c2ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f998dffb700 0 client.25177 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 shutdown_connections 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 1 -- 172.21.15.64:0/2976720441 wait complete. 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.818+0000 7f99b85b1b80 20 client.25177 trim_cache size 0 max 16384 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f99900a1270 msgr2=0x7f99900a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f99900a1270 0x7f99900a3730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f99a00095c0 tx=0x7f99a00d16f0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 msgr2=0x7f99a410cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a410cc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99a0006010 tx=0x7f99a000f510 comp rx=0 tx=0).stop 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 shutdown_connections 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f99900a1270 0x7f99900a3730 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4101110 0x7f99a410b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a40ff580 0x7f99a410cc60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 --2- 172.21.15.64:0/2774480604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99a4100770 0x7f99a410b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.819+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 >> 172.21.15.64:0/2774480604 conn(0x7f99a40faaf0 msgr2=0x7f99a4103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.820+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 shutdown_connections 2024-05-08T21:57:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:43.820+0000 7f99ac3f7700 1 -- 172.21.15.64:0/2774480604 wait complete. 2024-05-08T21:57:45.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:45 smithi186 ceph-mon[46619]: pgmap v353: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 MiB/s wr, 2.14k op/s 2024-05-08T21:57:45.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:45 smithi064 ceph-mon[42500]: pgmap v353: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 MiB/s wr, 2.14k op/s 2024-05-08T21:57:45.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:45 smithi160 ceph-mon[46962]: pgmap v353: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 MiB/s wr, 2.14k op/s 2024-05-08T21:57:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 175.0 s, 3032.5 tps, lat 33.178 ms stddev 35.822 2024-05-08T21:57:47.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:47 smithi186 ceph-mon[46619]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 2.00k op/s 2024-05-08T21:57:47.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:47 smithi064 ceph-mon[42500]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 2.00k op/s 2024-05-08T21:57:47.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:47 smithi160 ceph-mon[46962]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 2.00k op/s 2024-05-08T21:57:49.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:49 smithi186 ceph-mon[46619]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 1.94k op/s 2024-05-08T21:57:49.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:49 smithi064 ceph-mon[42500]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 1.94k op/s 2024-05-08T21:57:49.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:49 smithi160 ceph-mon[46962]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 1.94k op/s 2024-05-08T21:57:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 180.0 s, 2708.6 tps, lat 36.839 ms stddev 44.491 2024-05-08T21:57:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:51 smithi064 ceph-mon[42500]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 1.53k op/s 2024-05-08T21:57:51.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:51 smithi160 ceph-mon[46962]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 1.53k op/s 2024-05-08T21:57:51.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:51 smithi186 ceph-mon[46619]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 1.53k op/s 2024-05-08T21:57:53.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:53 smithi064 ceph-mon[42500]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 878 op/s 2024-05-08T21:57:53.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:53 smithi160 ceph-mon[46962]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 878 op/s 2024-05-08T21:57:53.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:53 smithi186 ceph-mon[46619]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 878 op/s 2024-05-08T21:57:55.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:55 smithi160 ceph-mon[46962]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 883 op/s 2024-05-08T21:57:55.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:55 smithi186 ceph-mon[46619]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 883 op/s 2024-05-08T21:57:55.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:55 smithi064 ceph-mon[42500]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 883 op/s 2024-05-08T21:57:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 185.0 s, 2043.8 tps, lat 48.289 ms stddev 54.408 2024-05-08T21:57:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 -- 172.21.15.64:0/855417593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c1020e0 msgr2=0x7f774c104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 --2- 172.21.15.64:0/855417593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c1020e0 0x7f774c104570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7748009990 tx=0x7f7748009ca0 comp rx=0 tx=0).stop 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 -- 172.21.15.64:0/855417593 shutdown_connections 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 --2- 172.21.15.64:0/855417593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c104ab0 0x7f774c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 --2- 172.21.15.64:0/855417593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f774c0ff770 0x7f774c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 --2- 172.21.15.64:0/855417593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c1020e0 0x7f774c104570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 -- 172.21.15.64:0/855417593 >> 172.21.15.64:0/855417593 conn(0x7f774c0fb390 msgr2=0x7f774c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.988+0000 7f7753a81700 1 -- 172.21.15.64:0/855417593 shutdown_connections 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 -- 172.21.15.64:0/855417593 wait complete. 2024-05-08T21:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 Processor -- start 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 -- start start 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 0x7f774c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c1020e0 0x7f774c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f774c104ab0 0x7f774c1a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c1a5000 con 0x7f774c1020e0 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c06a9c0 con 0x7f774c0ff770 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.989+0000 7f7753a81700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c06ab50 con 0x7f774c104ab0 2024-05-08T21:57:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 0x7f774c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 0x7f774c19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:46624/0 (socket says 172.21.15.64:46624) 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 -- 172.21.15.64:0/694947425 learned_addr learned my addr 172.21.15.64:0/694947425 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 -- 172.21.15.64:0/694947425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f774c104ab0 msgr2=0x7f774c1a4a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f774c104ab0 0x7f774c1a4a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 -- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c1020e0 msgr2=0x7f774c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c1020e0 0x7f774c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.990+0000 7f775181d700 1 -- 172.21.15.64:0/694947425 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480095f0 con 0x7f774c0ff770 2024-05-08T21:57:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.992+0000 7f775181d700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 0x7f774c19d4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f773c00d950 tx=0x7f773c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.992+0000 7f7742ffd700 1 -- 172.21.15.64:0/694947425 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c009b40 con 0x7f774c0ff770 2024-05-08T21:57:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.992+0000 7f7742ffd700 1 -- 172.21.15.64:0/694947425 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c009ca0 con 0x7f774c0ff770 2024-05-08T21:57:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.992+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c06ae30 con 0x7f774c0ff770 2024-05-08T21:57:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.992+0000 7f7742ffd700 1 -- 172.21.15.64:0/694947425 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00f620 con 0x7f774c0ff770 2024-05-08T21:57:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.992+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c06b390 con 0x7f774c0ff770 2024-05-08T21:57:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.995+0000 7f7742ffd700 1 -- 172.21.15.64:0/694947425 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f773c010460 con 0x7f774c0ff770 2024-05-08T21:57:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f7742ffd700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f77380a11a0 0x7f77380a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775101c700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f77380a11a0 0x7f77380a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f7742ffd700 1 -- 172.21.15.64:0/694947425 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f773c0cc7b0 con 0x7f774c0ff770 2024-05-08T21:57:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 Processor -- start 2024-05-08T21:57:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 -- start start 2024-05-08T21:57:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 0x55ae7cabfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cac1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7ca170f0 con 0x55ae7cac12d0 2024-05-08T21:57:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7ca14ab0 con 0x55ae7cabfa80 2024-05-08T21:57:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7ca103a0 con 0x55ae7cac03e0 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.996+0000 7f775101c700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f77380a11a0 0x7f77380a3660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f77480095c0 tx=0x7f7748009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 0x55ae7cabfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 0x55ae7cabfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:46632/0 (socket says 172.21.15.64:46632) 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 -- 172.21.15.64:0/2013281584 learned_addr learned my addr 172.21.15.64:0/2013281584 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775101c700 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775201e700 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cac1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 -- 172.21.15.64:0/2013281584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 msgr2=0x55ae7cac0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 -- 172.21.15.64:0/2013281584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 msgr2=0x55ae7cac1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cac1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 -- 172.21.15.64:0/2013281584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae7c891350 con 0x55ae7cabfa80 2024-05-08T21:57:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.997+0000 7f775181d700 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 0x55ae7cabfea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f773c005950 tx=0x7f773c00def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287861e54ce52801 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.998+0000 7f7737fff700 1 -- 172.21.15.64:0/2013281584 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c09da40 con 0x55ae7cabfa80 2024-05-08T21:57:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.998+0000 7f7737fff700 1 -- 172.21.15.64:0/2013281584 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c0a2090 con 0x55ae7cabfa80 2024-05-08T21:57:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.998+0000 7f7737fff700 1 -- 172.21.15.64:0/2013281584 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c0a1e60 con 0x55ae7cabfa80 2024-05-08T21:57:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2013281584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 msgr2=0x55ae7cabfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 0x55ae7cabfea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f773c005950 tx=0x7f773c00def0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2013281584 shutdown_connections 2024-05-08T21:57:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cabfa80 0x55ae7cabfea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cac1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2013281584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2013281584 >> 172.21.15.64:0/2013281584 conn(0x55ae7ca1e1b0 msgr2=0x55ae7cabc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2013281584 shutdown_connections 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:56.999+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2013281584 wait complete. 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 -- start start 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 Processor -- start 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cabfa80 0x55ae7cac98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cad0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7c9ac500 con 0x55ae7cabfa80 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7c9ac1a0 con 0x55ae7cac12d0 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7c893640 con 0x55ae7cac03e0 2024-05-08T21:57:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.000+0000 7f775fc3bb80 10 client.0 _ll_register_callbacks cb 0x55ae7c9b5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.001+0000 7f775201e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cad0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.001+0000 7f775201e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cad0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:46640/0 (socket says 172.21.15.64:46640) 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.001+0000 7f775201e700 1 -- 172.21.15.64:0/2988424575 learned_addr learned my addr 172.21.15.64:0/2988424575 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.001+0000 7f775101c700 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.001+0000 7f775201e700 1 -- 172.21.15.64:0/2988424575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 msgr2=0x55ae7cac9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.001+0000 7f775201e700 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775201e700 1 -- 172.21.15.64:0/2988424575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cabfa80 msgr2=0x55ae7cac98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775201e700 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cabfa80 0x55ae7cac98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775201e700 1 -- 172.21.15.64:0/2988424575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c06b390 con 0x55ae7cac12d0 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775201e700 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cad0e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7744009610 tx=0x7f7744004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897e83eb7c15fd27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f77357fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f77357fa700 1 -- 172.21.15.64:0/2988424575 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744007910 con 0x55ae7cac12d0 2024-05-08T21:57:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f77357fa700 1 -- 172.21.15.64:0/2988424575 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7744007a70 con 0x55ae7cac12d0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae7c969210 con 0x55ae7cac12d0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f77357fa700 1 -- 172.21.15.64:0/2988424575 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7744007c30 con 0x55ae7cac12d0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775fc3bb80 10 client.25271 fetch_fsmap learned FSMap version 35 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.002+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ae7c83db00 con 0x55ae7cac12d0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f77357fa700 1 -- 172.21.15.64:0/2988424575 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae7c969210 con 0x55ae7cac12d0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f77357fa700 1 -- 172.21.15.64:0/2988424575 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f774401e510 con 0x55ae7cac12d0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f775fc3bb80 10 client.25271 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f775fc3bb80 10 client.25271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f775fc3bb80 20 client.25271 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cad5820 0x55ae7cad7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f775fc3bb80 4 client.25271 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.003+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae7ca8f7b0 con 0x55ae7cad5820 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.004+0000 7f775181d700 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cad5820 0x55ae7cad7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.004+0000 7f77357fa700 10 client.25271 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.004+0000 7f775181d700 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cad5820 0x55ae7cad7c40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cfe020b5c9c3cc server_cookie=4678a7c59cbbde79 in_seq=0 out_seq=0 2024-05-08T21:57:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.005+0000 7f77357fa700 1 -- 172.21.15.64:0/2988424575 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae7ca8f7b0 con 0x55ae7cad5820 2024-05-08T21:57:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.005+0000 7f77357fa700 10 client.25271 handle_command_reply: tid=0 2024-05-08T21:57:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.005+0000 7f775fc3bb80 1 client.25271 shutdown 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 msgr2=0x55ae7cad0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cad0e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7744009610 tx=0x7f7744004320 comp rx=0 tx=0).stop 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 shutdown_connections 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cad0e30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cad5820 0x55ae7cad7c40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cabfa80 0x55ae7cac98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2988424575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cac03e0 0x55ae7cac9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 >> 172.21.15.64:0/2988424575 conn(0x55ae7ca1e1b0 msgr2=0x55ae7caba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f77357fa700 0 client.25271 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 shutdown_connections 2024-05-08T21:57:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2988424575 wait complete. 2024-05-08T21:57:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.006+0000 7f775fc3bb80 20 client.25271 trim_cache size 0 max 16384 2024-05-08T21:57:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.034+0000 7f775fc3bb80 1 Processor -- start 2024-05-08T21:57:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 -- start start 2024-05-08T21:57:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cad14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 0x55ae7cad0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cac16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cac0f00 con 0x55ae7cac03e0 2024-05-08T21:57:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cac1100 con 0x55ae7cac12d0 2024-05-08T21:57:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.035+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cacaaa0 con 0x55ae7cabfa80 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.036+0000 7f775101c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 0x55ae7cad0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.036+0000 7f775101c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 0x55ae7cad0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36016/0 (socket says 172.21.15.64:36016) 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.036+0000 7f775101c700 1 -- 172.21.15.64:0/627874878 learned_addr learned my addr 172.21.15.64:0/627874878 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775181d700 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cad14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775181d700 1 -- 172.21.15.64:0/627874878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 msgr2=0x55ae7cac16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775181d700 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cac16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775181d700 1 -- 172.21.15.64:0/627874878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 msgr2=0x55ae7cad0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775181d700 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 0x55ae7cad0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775181d700 1 -- 172.21.15.64:0/627874878 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae7c9d6500 con 0x55ae7cabfa80 2024-05-08T21:57:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.037+0000 7f775101c700 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 0x55ae7cad0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:57:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.038+0000 7f775181d700 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cad14e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f773c096110 tx=0x7f773c099100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4a9e7030a727cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.038+0000 7f7740da4700 1 -- 172.21.15.64:0/627874878 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c09dac0 con 0x55ae7cabfa80 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.038+0000 7f7740da4700 1 -- 172.21.15.64:0/627874878 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c0991e0 con 0x55ae7cabfa80 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.039+0000 7f7740da4700 1 -- 172.21.15.64:0/627874878 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c010710 con 0x55ae7cabfa80 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.039+0000 7f775fc3bb80 1 -- 172.21.15.64:0/627874878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 msgr2=0x55ae7cad14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.039+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cad14e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f773c096110 tx=0x7f773c099100 comp rx=0 tx=0).stop 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.040+0000 7f775fc3bb80 1 -- 172.21.15.64:0/627874878 shutdown_connections 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.040+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac12d0 0x55ae7cac16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.040+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cad14e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.040+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/627874878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac03e0 0x55ae7cad0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.040+0000 7f775fc3bb80 1 -- 172.21.15.64:0/627874878 >> 172.21.15.64:0/627874878 conn(0x55ae7ca1e1b0 msgr2=0x55ae7cac7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.042+0000 7f775fc3bb80 1 -- 172.21.15.64:0/627874878 shutdown_connections 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.042+0000 7f775fc3bb80 1 -- 172.21.15.64:0/627874878 wait complete. 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 -- start start 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 Processor -- start 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cace070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac03e0 0x55ae7cace5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cb4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cb4c250 con 0x55ae7cac12d0 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cb4c390 con 0x55ae7cac03e0 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cb4c500 con 0x55ae7cabfa80 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.043+0000 7f775fc3bb80 10 client.0 _ll_register_callbacks cb 0x55ae7c9b5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:57:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cb4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cb4bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36018/0 (socket says 172.21.15.64:36018) 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 -- 172.21.15.64:0/2480939212 learned_addr learned my addr 172.21.15.64:0/2480939212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 -- 172.21.15.64:0/2480939212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 msgr2=0x55ae7cace070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cace070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 -- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac03e0 msgr2=0x55ae7cace5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac03e0 0x55ae7cace5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.044+0000 7f775201e700 1 -- 172.21.15.64:0/2480939212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae7c891350 con 0x55ae7cac12d0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.045+0000 7f775201e700 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cb4bd10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f774401e8a0 tx=0x7f7744014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a284c48b0a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.045+0000 7f77357fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.045+0000 7f77357fa700 1 -- 172.21.15.64:0/2480939212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774401eb90 con 0x55ae7cac12d0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.045+0000 7f77357fa700 1 -- 172.21.15.64:0/2480939212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77440049f0 con 0x55ae7cac12d0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.045+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f773c0a2090 con 0x55ae7cac12d0 2024-05-08T21:57:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.046+0000 7f77357fa700 1 -- 172.21.15.64:0/2480939212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7744004bb0 con 0x55ae7cac12d0 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.047+0000 7f775fc3bb80 10 client.15543 fetch_fsmap learned FSMap version 35 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.047+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ae7c895c70 con 0x55ae7cac12d0 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f77357fa700 1 -- 172.21.15.64:0/2480939212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c0a2090 con 0x55ae7cac12d0 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f77357fa700 1 -- 172.21.15.64:0/2480939212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7744028070 con 0x55ae7cac12d0 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f775fc3bb80 10 client.15543 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f775fc3bb80 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f775fc3bb80 20 client.15543 populate_metadata read hostname 'smithi064' 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cb4dfa0 0x55ae7cb503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f775fc3bb80 4 client.15543 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:57:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.049+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae7ca08e60 con 0x55ae7cb4dfa0 2024-05-08T21:57:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.050+0000 7f775181d700 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cb4dfa0 0x55ae7cb503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:57:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.050+0000 7f775181d700 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cb4dfa0 0x55ae7cb503e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3357c389406e9a36 server_cookie=c1dc5f94ce9ff5c4 in_seq=0 out_seq=0 2024-05-08T21:57:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.050+0000 7f77357fa700 10 client.15543 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f77357fa700 1 -- 172.21.15.64:0/2480939212 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae7ca08e60 con 0x55ae7cb4dfa0 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f77357fa700 10 client.15543 handle_command_reply: tid=0 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 client.15543 shutdown 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 msgr2=0x55ae7cb4bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cb4bd10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f774401e8a0 tx=0x7f7744014000 comp rx=0 tx=0).stop 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 shutdown_connections 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ae7cb4dfa0 0x55ae7cb503e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae7cac12d0 0x55ae7cb4bd10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae7cabfa80 0x55ae7cace070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 --2- 172.21.15.64:0/2480939212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae7cac03e0 0x55ae7cace5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.068+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 >> 172.21.15.64:0/2480939212 conn(0x55ae7ca1e1b0 msgr2=0x55ae7cadc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.069+0000 7f77357fa700 0 client.15543 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.069+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 shutdown_connections 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.069+0000 7f775fc3bb80 1 -- 172.21.15.64:0/2480939212 wait complete. 2024-05-08T21:57:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.070+0000 7f775fc3bb80 20 client.15543 trim_cache size 0 max 16384 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.071+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f77380a11a0 msgr2=0x7f77380a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.071+0000 7f7753a81700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f77380a11a0 0x7f77380a3660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f77480095c0 tx=0x7f7748009550 comp rx=0 tx=0).stop 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.071+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 msgr2=0x7f774c19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:57:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.071+0000 7f7753a81700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 0x7f774c19d4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f773c00d950 tx=0x7f773c00dd10 comp rx=0 tx=0).stop 2024-05-08T21:57:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 shutdown_connections 2024-05-08T21:57:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f77380a11a0 0x7f77380a3660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f774c0ff770 0x7f774c19d4d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f774c104ab0 0x7f774c1a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 --2- 172.21.15.64:0/694947425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c1020e0 0x7f774c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:57:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 >> 172.21.15.64:0/694947425 conn(0x7f774c0fb390 msgr2=0x7f774c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:57:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 shutdown_connections 2024-05-08T21:57:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:57:57.072+0000 7f7753a81700 1 -- 172.21.15.64:0/694947425 wait complete. 2024-05-08T21:57:57.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:57 smithi160 ceph-mon[46962]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 809 op/s 2024-05-08T21:57:57.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:57 smithi186 ceph-mon[46619]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 809 op/s 2024-05-08T21:57:57.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:57 smithi064 ceph-mon[42500]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 809 op/s 2024-05-08T21:57:59.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:57:59 smithi160 ceph-mon[46962]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 880 op/s 2024-05-08T21:57:59.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:57:59 smithi186 ceph-mon[46619]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 880 op/s 2024-05-08T21:57:59.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:57:59 smithi064 ceph-mon[42500]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 880 op/s 2024-05-08T21:58:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 190.0 s, 2038.8 tps, lat 49.629 ms stddev 56.392 2024-05-08T21:58:01.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:01 smithi160 ceph-mon[46962]: pgmap v361: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2024-05-08T21:58:01.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:01 smithi186 ceph-mon[46619]: pgmap v361: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2024-05-08T21:58:01.969 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:01 smithi064 ceph-mon[42500]: pgmap v361: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2024-05-08T21:58:03.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:03 smithi160 ceph-mon[46962]: pgmap v362: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.06k op/s 2024-05-08T21:58:03.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:03 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:03.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:03 smithi186 ceph-mon[46619]: pgmap v362: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.06k op/s 2024-05-08T21:58:03.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:03 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:03.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:03 smithi064 ceph-mon[42500]: pgmap v362: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.06k op/s 2024-05-08T21:58:03.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:03 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:05.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:05 smithi186 ceph-mon[46619]: pgmap v363: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 1.53k op/s 2024-05-08T21:58:05.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:05 smithi064 ceph-mon[42500]: pgmap v363: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 1.53k op/s 2024-05-08T21:58:06.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:05 smithi160 ceph-mon[46962]: pgmap v363: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 1.53k op/s 2024-05-08T21:58:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 195.0 s, 1904.6 tps, lat 52.629 ms stddev 80.151 2024-05-08T21:58:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:07 smithi186 ceph-mon[46619]: pgmap v364: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-05-08T21:58:07.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:07 smithi064 ceph-mon[42500]: pgmap v364: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-05-08T21:58:08.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:07 smithi160 ceph-mon[46962]: pgmap v364: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-05-08T21:58:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:09 smithi186 ceph-mon[46619]: pgmap v365: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 MiB/s wr, 2.18k op/s 2024-05-08T21:58:09.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:09 smithi064 ceph-mon[42500]: pgmap v365: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 MiB/s wr, 2.18k op/s 2024-05-08T21:58:10.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:09 smithi160 ceph-mon[46962]: pgmap v365: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 MiB/s wr, 2.18k op/s 2024-05-08T21:58:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.252+0000 7f011f018700 1 -- 172.21.15.64:0/1860861560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180ff770 msgr2=0x7f0118101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.252+0000 7f011f018700 1 --2- 172.21.15.64:0/1860861560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180ff770 0x7f0118101ba0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0108009990 tx=0x7f0108009ca0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 -- 172.21.15.64:0/1860861560 shutdown_connections 2024-05-08T21:58:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 --2- 172.21.15.64:0/1860861560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f0118104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 --2- 172.21.15.64:0/1860861560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180ff770 0x7f0118101ba0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 --2- 172.21.15.64:0/1860861560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01181020e0 0x7f0118104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 -- 172.21.15.64:0/1860861560 >> 172.21.15.64:0/1860861560 conn(0x7f01180fb390 msgr2=0x7f01180fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 -- 172.21.15.64:0/1860861560 shutdown_connections 2024-05-08T21:58:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.253+0000 7f011f018700 1 -- 172.21.15.64:0/1860861560 wait complete. 2024-05-08T21:58:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 Processor -- start 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 -- start start 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01180ff770 0x7f011806f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01181020e0 0x7f011806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f01181ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118072e50 con 0x7f01181020e0 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118072f90 con 0x7f01180ff770 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011f018700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118105d90 con 0x7f0118104ab0 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011d5b5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f01181ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011d5b5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f01181ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48234/0 (socket says 172.21.15.64:48234) 2024-05-08T21:58:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.254+0000 7f011d5b5700 1 -- 172.21.15.64:0/3891222507 learned_addr learned my addr 172.21.15.64:0/3891222507 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.255+0000 7f011d5b5700 1 -- 172.21.15.64:0/3891222507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01180ff770 msgr2=0x7f011806f880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:58:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.255+0000 7f011d5b5700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01180ff770 0x7f011806f880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.255+0000 7f011d5b5700 1 -- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01181020e0 msgr2=0x7f011806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.255+0000 7f011d5b5700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01181020e0 0x7f011806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.255+0000 7f011d5b5700 1 -- 172.21.15.64:0/3891222507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080095f0 con 0x7f0118104ab0 2024-05-08T21:58:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.256+0000 7f011d5b5700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f01181ad5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f011000e940 tx=0x7f011000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.256+0000 7f010dffb700 1 -- 172.21.15.64:0/3891222507 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01100097b0 con 0x7f0118104ab0 2024-05-08T21:58:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.256+0000 7f010dffb700 1 -- 172.21.15.64:0/3891222507 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0110004cf0 con 0x7f0118104ab0 2024-05-08T21:58:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.256+0000 7f010dffb700 1 -- 172.21.15.64:0/3891222507 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01100055d0 con 0x7f0118104ab0 2024-05-08T21:58:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.257+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0118106070 con 0x7f0118104ab0 2024-05-08T21:58:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.257+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0118106530 con 0x7f0118104ab0 2024-05-08T21:58:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.261+0000 7f012b1d2b80 1 Processor -- start 2024-05-08T21:58:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.262+0000 7f010dffb700 1 -- 172.21.15.64:0/3891222507 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f011000cbd0 con 0x7f0118104ab0 2024-05-08T21:58:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.262+0000 7f012b1d2b80 1 -- start start 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.262+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.262+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa0fa0f0 con 0x55a9aa1a2a80 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa0f7ab0 con 0x55a9aa1a42d0 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa0f33a0 con 0x55a9aa1a33e0 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f011cdb4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f010ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f011cdb4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48236/0 (socket says 172.21.15.64:48236) 2024-05-08T21:58:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f010ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45654/0 (socket says 172.21.15.64:45654) 2024-05-08T21:58:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.263+0000 7f011cdb4700 1 -- 172.21.15.64:0/2323700655 learned_addr learned my addr 172.21.15.64:0/2323700655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.264+0000 7f010dffb700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f01000a12f0 0x7f01000a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.264+0000 7f010dffb700 1 -- 172.21.15.64:0/3891222507 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0110014070 con 0x7f0118104ab0 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.264+0000 7f010ffff700 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.265+0000 7f010ffff700 1 -- 172.21.15.64:0/2323700655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a33e0 msgr2=0x55a9aa1a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.265+0000 7f010ffff700 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.265+0000 7f010ffff700 1 -- 172.21.15.64:0/2323700655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 msgr2=0x55a9aa1a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.265+0000 7f010ffff700 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.265+0000 7f010ffff700 1 -- 172.21.15.64:0/2323700655 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9a9f74350 con 0x55a9aa1a42d0 2024-05-08T21:58:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.266+0000 7f010ffff700 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a4750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0114009c10 tx=0x7f0114009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ecd0145cee9917b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.266+0000 7f011d5b5700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f01000a12f0 0x7f01000a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.266+0000 7f00feffd700 1 -- 172.21.15.64:0/2323700655 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114008db0 con 0x55a9aa1a42d0 2024-05-08T21:58:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.266+0000 7f00feffd700 1 -- 172.21.15.64:0/2323700655 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0114004bb0 con 0x55a9aa1a42d0 2024-05-08T21:58:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.266+0000 7f00feffd700 1 -- 172.21.15.64:0/2323700655 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114005020 con 0x55a9aa1a42d0 2024-05-08T21:58:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 -- 172.21.15.64:0/2323700655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 msgr2=0x55a9aa1a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a4750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0114009c10 tx=0x7f0114009eb0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f011d5b5700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f01000a12f0 0x7f01000a37b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0110099e00 tx=0x7f0110096280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 -- 172.21.15.64:0/2323700655 shutdown_connections 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a4750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/2323700655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 -- 172.21.15.64:0/2323700655 >> 172.21.15.64:0/2323700655 conn(0x55a9aa1011b0 msgr2=0x55a9aa19f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 -- 172.21.15.64:0/2323700655 shutdown_connections 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 -- 172.21.15.64:0/2323700655 wait complete. 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.267+0000 7f012b1d2b80 1 -- start start 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 Processor -- start 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa08f500 con 0x55a9aa1a42d0 2024-05-08T21:58:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa08f1a0 con 0x55a9aa1a33e0 2024-05-08T21:58:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a9f76640 con 0x55a9aa1a2a80 2024-05-08T21:58:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f012b1d2b80 10 client.0 _ll_register_callbacks cb 0x55a9aa098ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f010ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f010ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57900/0 (socket says 172.21.15.64:57900) 2024-05-08T21:58:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f010ffff700 1 -- 172.21.15.64:0/1277482423 learned_addr learned my addr 172.21.15.64:0/1277482423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f010ffff700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.268+0000 7f011cdb4700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.269+0000 7f011cdb4700 1 -- 172.21.15.64:0/1277482423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 msgr2=0x55a9aa1ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.269+0000 7f011cdb4700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.269+0000 7f011cdb4700 1 -- 172.21.15.64:0/1277482423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 msgr2=0x55a9aa1b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.269+0000 7f011cdb4700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.269+0000 7f011cdb4700 1 -- 172.21.15.64:0/1277482423 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0114011040 con 0x55a9aa1a33e0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f011cdb4700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1ace60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0108006010 tx=0x7f010800bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78cdf3524f3af6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f00f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f00f7fff700 1 -- 172.21.15.64:0/1277482423 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010801f040 con 0x55a9aa1a33e0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f00f7fff700 1 -- 172.21.15.64:0/1277482423 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0108004500 con 0x55a9aa1a33e0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9aa04c210 con 0x55a9aa1a33e0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f00f7fff700 1 -- 172.21.15.64:0/1277482423 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0108005a00 con 0x55a9aa1a33e0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f012b1d2b80 10 client.25280 fetch_fsmap learned FSMap version 36 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.270+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a9a9f20b00 con 0x55a9aa1a33e0 2024-05-08T21:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.271+0000 7f00f7fff700 1 -- 172.21.15.64:0/1277482423 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9aa04c210 con 0x55a9aa1a33e0 2024-05-08T21:58:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f00f7fff700 1 -- 172.21.15.64:0/1277482423 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f01080046c0 con 0x55a9aa1a33e0 2024-05-08T21:58:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f012b1d2b80 10 client.25280 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T21:58:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f012b1d2b80 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f012b1d2b80 20 client.25280 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa1b8890 0x55a9aa1bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f012b1d2b80 4 client.25280 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9aa1727b0 con 0x55a9aa1b8890 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.272+0000 7f010ffff700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa1b8890 0x55a9aa1bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.273+0000 7f010ffff700 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa1b8890 0x55a9aa1bacb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d988461fa0f737 server_cookie=1d3e21aa3f305a6f in_seq=0 out_seq=0 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.273+0000 7f00f7fff700 10 client.25280 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.274+0000 7f00f7fff700 1 -- 172.21.15.64:0/1277482423 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9aa1727b0 con 0x55a9aa1b8890 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.274+0000 7f00f7fff700 10 client.25280 handle_command_reply: tid=0 2024-05-08T21:58:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.274+0000 7f012b1d2b80 1 client.25280 shutdown 2024-05-08T21:58:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 msgr2=0x55a9aa1ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1ace60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0108006010 tx=0x7f010800bef0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 shutdown_connections 2024-05-08T21:58:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa1b8890 0x55a9aa1bacb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/1277482423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1ace60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f00f7fff700 0 client.25280 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.275+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 >> 172.21.15.64:0/1277482423 conn(0x55a9aa1011b0 msgr2=0x55a9aa19d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.276+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 shutdown_connections 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.276+0000 7f012b1d2b80 1 -- 172.21.15.64:0/1277482423 wait complete. 2024-05-08T21:58:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.276+0000 7f012b1d2b80 20 client.25280 trim_cache size 0 max 16384 2024-05-08T21:58:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 Processor -- start 2024-05-08T21:58:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 -- start start 2024-05-08T21:58:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa1ac490 con 0x55a9aa1a2a80 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa19e930 con 0x55a9aa1a33e0 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.297+0000 7f012b1d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa1ab390 con 0x55a9aa1a42d0 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45670/0 (socket says 172.21.15.64:45670) 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 -- 172.21.15.64:0/538872838 learned_addr learned my addr 172.21.15.64:0/538872838 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 -- 172.21.15.64:0/538872838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a42d0 msgr2=0x55a9aa1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 -- 172.21.15.64:0/538872838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 msgr2=0x55a9aa1bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 -- 172.21.15.64:0/538872838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9aa0b9500 con 0x55a9aa1a33e0 2024-05-08T21:58:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.298+0000 7f011cdb4700 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b3e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0108009330 tx=0x7f01080058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af01155a249c4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.299+0000 7f00ff7fe700 1 -- 172.21.15.64:0/538872838 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010801f040 con 0x55a9aa1a33e0 2024-05-08T21:58:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.299+0000 7f00ff7fe700 1 -- 172.21.15.64:0/538872838 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0108005bc0 con 0x55a9aa1a33e0 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.299+0000 7f00ff7fe700 1 -- 172.21.15.64:0/538872838 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01080247b0 con 0x55a9aa1a33e0 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.299+0000 7f012b1d2b80 1 -- 172.21.15.64:0/538872838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 msgr2=0x55a9aa1b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.299+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b3e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0108009330 tx=0x7f01080058e0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 -- 172.21.15.64:0/538872838 shutdown_connections 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a42d0 0x55a9aa1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/538872838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b3e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 -- 172.21.15.64:0/538872838 >> 172.21.15.64:0/538872838 conn(0x55a9aa1011b0 msgr2=0x55a9aa1a7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 -- 172.21.15.64:0/538872838 shutdown_connections 2024-05-08T21:58:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 -- 172.21.15.64:0/538872838 wait complete. 2024-05-08T21:58:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 -- start start 2024-05-08T21:58:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.300+0000 7f012b1d2b80 1 Processor -- start 2024-05-08T21:58:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.301+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.301+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.302+0000 7f010ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.302+0000 7f010ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48268/0 (socket says 172.21.15.64:48268) 2024-05-08T21:58:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f012b1d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa20cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa1b1860 con 0x55a9aa1a42d0 2024-05-08T21:58:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f010ffff700 1 -- 172.21.15.64:0/3915053234 learned_addr learned my addr 172.21.15.64:0/3915053234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa20d270 con 0x55a9aa1a33e0 2024-05-08T21:58:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aa20d3b0 con 0x55a9aa1a2a80 2024-05-08T21:58:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f012b1d2b80 10 client.0 _ll_register_callbacks cb 0x55a9aa098ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.303+0000 7f010ffff700 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa20cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f010ffff700 1 -- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 msgr2=0x55a9aa1b1240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f010ffff700 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b1240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f010ffff700 1 -- 172.21.15.64:0/3915053234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 msgr2=0x55a9aa20cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f010ffff700 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa20cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f010ffff700 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9a9f20b00 con 0x55a9aa1a2a80 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f010ffff700 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1b0d00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0114009ef0 tx=0x7f01140098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec3d3f8fa160ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f00f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f00f7fff700 1 -- 172.21.15.64:0/3915053234 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114003a80 con 0x55a9aa1a2a80 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f00f7fff700 1 -- 172.21.15.64:0/3915053234 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0114003be0 con 0x55a9aa1a2a80 2024-05-08T21:58:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.304+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0114004bb0 con 0x55a9aa1a2a80 2024-05-08T21:58:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.307+0000 7f00f7fff700 1 -- 172.21.15.64:0/3915053234 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0114009990 con 0x55a9aa1a2a80 2024-05-08T21:58:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.307+0000 7f012b1d2b80 10 client.25204 fetch_fsmap learned FSMap version 36 2024-05-08T21:58:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.307+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a9a9f78c70 con 0x55a9aa1a2a80 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f00f7fff700 1 -- 172.21.15.64:0/3915053234 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0114004bb0 con 0x55a9aa1a2a80 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f00f7fff700 1 -- 172.21.15.64:0/3915053234 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0114004040 con 0x55a9aa1a2a80 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f012b1d2b80 10 client.25204 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f012b1d2b80 10 client.25204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f012b1d2b80 20 client.25204 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa20ec20 0x55a9aa211040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f012b1d2b80 4 client.25204 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.308+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9aa0ebe60 con 0x55a9aa20ec20 2024-05-08T21:58:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.310+0000 7f011cdb4700 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa20ec20 0x55a9aa211040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.310+0000 7f00f7fff700 10 client.25204 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.310+0000 7f011cdb4700 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa20ec20 0x55a9aa211040 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581dd9babf0d020a server_cookie=70edb36ac7875a03 in_seq=0 out_seq=0 2024-05-08T21:58:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.323+0000 7f00f7fff700 1 -- 172.21.15.64:0/3915053234 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9aa0ebe60 con 0x55a9aa20ec20 2024-05-08T21:58:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.323+0000 7f00f7fff700 10 client.25204 handle_command_reply: tid=0 2024-05-08T21:58:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.323+0000 7f012b1d2b80 1 client.25204 shutdown 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 msgr2=0x55a9aa1b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1b0d00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0114009ef0 tx=0x7f01140098a0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 shutdown_connections 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a9aa20ec20 0x55a9aa211040 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9aa1a42d0 0x55a9aa20cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9aa1a2a80 0x55a9aa1b0d00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f00f7fff700 0 client.25204 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 --2- 172.21.15.64:0/3915053234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a9aa1a33e0 0x55a9aa1b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 >> 172.21.15.64:0/3915053234 conn(0x55a9aa1011b0 msgr2=0x55a9aa1aa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 shutdown_connections 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 1 -- 172.21.15.64:0/3915053234 wait complete. 2024-05-08T21:58:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.324+0000 7f012b1d2b80 20 client.25204 trim_cache size 0 max 16384 2024-05-08T21:58:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:58:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:58:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:58:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:58:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f01000a12f0 msgr2=0x7f01000a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f01000a12f0 0x7f01000a37b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0110099e00 tx=0x7f0110096280 comp rx=0 tx=0).stop 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 msgr2=0x7f01181ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f01181ad5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f011000e940 tx=0x7f011000ed00 comp rx=0 tx=0).stop 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 shutdown_connections 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f01000a12f0 0x7f01000a37b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01180ff770 0x7f011806f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0118104ab0 0x7f01181ad5d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 --2- 172.21.15.64:0/3891222507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01181020e0 0x7f011806fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 >> 172.21.15.64:0/3891222507 conn(0x7f01180fb390 msgr2=0x7f0118102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 shutdown_connections 2024-05-08T21:58:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:10.327+0000 7f011f018700 1 -- 172.21.15.64:0/3891222507 wait complete. 2024-05-08T21:58:11.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 200.0 s, 2593.0 tps, lat 38.502 ms stddev 43.117 2024-05-08T21:58:11.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:11 smithi186 ceph-mon[46619]: pgmap v366: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 MiB/s wr, 2.23k op/s 2024-05-08T21:58:11.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:11 smithi064 ceph-mon[42500]: pgmap v366: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 MiB/s wr, 2.23k op/s 2024-05-08T21:58:12.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:11 smithi160 ceph-mon[46962]: pgmap v366: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 MiB/s wr, 2.23k op/s 2024-05-08T21:58:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:13 smithi186 ceph-mon[46619]: pgmap v367: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 47 MiB/s wr, 2.10k op/s 2024-05-08T21:58:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:13 smithi064 ceph-mon[42500]: pgmap v367: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 47 MiB/s wr, 2.10k op/s 2024-05-08T21:58:14.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:13 smithi160 ceph-mon[46962]: pgmap v367: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 47 MiB/s wr, 2.10k op/s 2024-05-08T21:58:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:15 smithi186 ceph-mon[46619]: pgmap v368: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 MiB/s wr, 2.02k op/s 2024-05-08T21:58:15.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:15 smithi064 ceph-mon[42500]: pgmap v368: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 MiB/s wr, 2.02k op/s 2024-05-08T21:58:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:15 smithi160 ceph-mon[46962]: pgmap v368: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 MiB/s wr, 2.02k op/s 2024-05-08T21:58:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 205.0 s, 2577.4 tps, lat 38.794 ms stddev 41.073 2024-05-08T21:58:17.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:17 smithi064 ceph-mon[42500]: pgmap v369: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 1.49k op/s 2024-05-08T21:58:18.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:17 smithi160 ceph-mon[46962]: pgmap v369: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 1.49k op/s 2024-05-08T21:58:18.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:17 smithi186 ceph-mon[46619]: pgmap v369: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 1.49k op/s 2024-05-08T21:58:19.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:19 smithi064 ceph-mon[42500]: pgmap v370: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.19k op/s 2024-05-08T21:58:20.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:19 smithi160 ceph-mon[46962]: pgmap v370: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.19k op/s 2024-05-08T21:58:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:19 smithi186 ceph-mon[46619]: pgmap v370: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.19k op/s 2024-05-08T21:58:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 210.0 s, 2629.3 tps, lat 37.934 ms stddev 43.713 2024-05-08T21:58:21.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:21 smithi064 ceph-mon[42500]: pgmap v371: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 840 op/s 2024-05-08T21:58:22.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:21 smithi160 ceph-mon[46962]: pgmap v371: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 840 op/s 2024-05-08T21:58:22.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:21 smithi186 ceph-mon[46619]: pgmap v371: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 840 op/s 2024-05-08T21:58:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.501+0000 7f05aa027700 1 -- 172.21.15.64:0/815504434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a41090d0 msgr2=0x7f05a410f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.501+0000 7f05aa027700 1 --2- 172.21.15.64:0/815504434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a41090d0 0x7f05a410f980 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0598009920 tx=0x7f0598009c30 comp rx=0 tx=0).stop 2024-05-08T21:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.509+0000 7f05aa027700 1 -- 172.21.15.64:0/815504434 shutdown_connections 2024-05-08T21:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.509+0000 7f05aa027700 1 --2- 172.21.15.64:0/815504434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a41090d0 0x7f05a410f980 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.509+0000 7f05aa027700 1 --2- 172.21.15.64:0/815504434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a4107540 0x7f05a4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.509+0000 7f05aa027700 1 --2- 172.21.15.64:0/815504434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4108730 0x7f05a4108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.509+0000 7f05aa027700 1 -- 172.21.15.64:0/815504434 >> 172.21.15.64:0/815504434 conn(0x7f05a40764d0 msgr2=0x7f05a4078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.521+0000 7f05aa027700 1 -- 172.21.15.64:0/815504434 shutdown_connections 2024-05-08T21:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.528+0000 7f05aa027700 1 -- 172.21.15.64:0/815504434 wait complete. 2024-05-08T21:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.528+0000 7f05aa027700 1 Processor -- start 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 -- start start 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4107540 0x7f05a4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4108730 0x7f05a4071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 0x7f05a4072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40725f0 con 0x7f05a4108730 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a4072760 con 0x7f05a4107540 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05aa027700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a4072920 con 0x7f05a41090d0 2024-05-08T21:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 0x7f05a4072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 0x7f05a4072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56052/0 (socket says 172.21.15.64:56052) 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.529+0000 7f05a3fff700 1 -- 172.21.15.64:0/3802280058 learned_addr learned my addr 172.21.15.64:0/3802280058 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.530+0000 7f05a3fff700 1 -- 172.21.15.64:0/3802280058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4107540 msgr2=0x7f05a4070af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.530+0000 7f05a3fff700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4107540 0x7f05a4070af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.530+0000 7f05a3fff700 1 -- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4108730 msgr2=0x7f05a4071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.530+0000 7f05a3fff700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4108730 0x7f05a4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.530+0000 7f05a3fff700 1 -- 172.21.15.64:0/3802280058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009520 con 0x7f05a41090d0 2024-05-08T21:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.530+0000 7f05a3fff700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 0x7f05a4072020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0598009230 tx=0x7f059800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.538+0000 7f05a0ff9700 1 -- 172.21.15.64:0/3802280058 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059801c070 con 0x7f05a41090d0 2024-05-08T21:58:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.538+0000 7f05a0ff9700 1 -- 172.21.15.64:0/3802280058 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f059800f840 con 0x7f05a41090d0 2024-05-08T21:58:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.538+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a4072b40 con 0x7f05a41090d0 2024-05-08T21:58:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.538+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a4072fa0 con 0x7f05a41090d0 2024-05-08T21:58:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.538+0000 7f05a0ff9700 1 -- 172.21.15.64:0/3802280058 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598004b70 con 0x7f05a41090d0 2024-05-08T21:58:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.540+0000 7f05a0ff9700 1 -- 172.21.15.64:0/3802280058 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0598020070 con 0x7f05a41090d0 2024-05-08T21:58:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.543+0000 7f05a0ff9700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f058c0a1320 0x7f058c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.543+0000 7f05a0ff9700 1 -- 172.21.15.64:0/3802280058 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05980cd5a0 con 0x7f05a41090d0 2024-05-08T21:58:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.543+0000 7f05a37fe700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f058c0a1320 0x7f058c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.544+0000 7f05a37fe700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f058c0a1320 0x7f058c0a37e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05a4108590 tx=0x7f059c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.544+0000 7f05b61e1b80 1 Processor -- start 2024-05-08T21:58:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.544+0000 7f05b61e1b80 1 -- start start 2024-05-08T21:58:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.545+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679de9a80 0x55b679de9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.545+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679dea3e0 0x55b679deacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.545+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679deb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.545+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679d410f0 con 0x55b679deb2d0 2024-05-08T21:58:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.545+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679d3eab0 con 0x55b679dea3e0 2024-05-08T21:58:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.545+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679d3a3a0 con 0x55b679de9a80 2024-05-08T21:58:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.548+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679deb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.548+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679deb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42226/0 (socket says 172.21.15.64:42226) 2024-05-08T21:58:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.548+0000 7f05a3fff700 1 -- 172.21.15.64:0/1055241416 learned_addr learned my addr 172.21.15.64:0/1055241416 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.549+0000 7f05a3fff700 1 -- 172.21.15.64:0/1055241416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679de9a80 msgr2=0x55b679de9ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:58:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.549+0000 7f05a3fff700 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679de9a80 0x55b679de9ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.549+0000 7f05a3fff700 1 -- 172.21.15.64:0/1055241416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679dea3e0 msgr2=0x55b679deacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.549+0000 7f05a3fff700 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679dea3e0 0x55b679deacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.549+0000 7f05a3fff700 1 -- 172.21.15.64:0/1055241416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b679bbb350 con 0x55b679deb2d0 2024-05-08T21:58:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.549+0000 7f05a3fff700 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679deb750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f05980974a0 tx=0x7f0598009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8b0c7885944a75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.550+0000 7f0589ffb700 1 -- 172.21.15.64:0/1055241416 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980a2e20 con 0x55b679deb2d0 2024-05-08T21:58:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.550+0000 7f0589ffb700 1 -- 172.21.15.64:0/1055241416 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05980cb160 con 0x55b679deb2d0 2024-05-08T21:58:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.550+0000 7f0589ffb700 1 -- 172.21.15.64:0/1055241416 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059809aef0 con 0x55b679deb2d0 2024-05-08T21:58:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 -- 172.21.15.64:0/1055241416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 msgr2=0x55b679deb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679deb750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f05980974a0 tx=0x7f0598009c30 comp rx=0 tx=0).stop 2024-05-08T21:58:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 -- 172.21.15.64:0/1055241416 shutdown_connections 2024-05-08T21:58:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679deb750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679de9a80 0x55b679de9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/1055241416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679dea3e0 0x55b679deacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.551+0000 7f05b61e1b80 1 -- 172.21.15.64:0/1055241416 >> 172.21.15.64:0/1055241416 conn(0x55b679d481b0 msgr2=0x55b679de6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.552+0000 7f05b61e1b80 1 -- 172.21.15.64:0/1055241416 shutdown_connections 2024-05-08T21:58:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.552+0000 7f05b61e1b80 1 -- 172.21.15.64:0/1055241416 wait complete. 2024-05-08T21:58:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.552+0000 7f05b61e1b80 1 -- start start 2024-05-08T21:58:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 Processor -- start 2024-05-08T21:58:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 0x55b679df3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df3da0 0x55b679df8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df4220 0x55b679dfb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679cd6500 con 0x55b679df3da0 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679cd61a0 con 0x55b679df4220 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679bbd640 con 0x55b679deb2d0 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05b61e1b80 10 client.0 _ll_register_callbacks cb 0x55b679cdfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.553+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 0x55b679df3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 0x55b679df3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56078/0 (socket says 172.21.15.64:56078) 2024-05-08T21:58:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 -- 172.21.15.64:0/2807521133 learned_addr learned my addr 172.21.15.64:0/2807521133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df3da0 0x55b679df8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a3fff700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df4220 0x55b679dfb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 -- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df4220 msgr2=0x55b679dfb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df4220 0x55b679dfb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 -- 172.21.15.64:0/2807521133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df3da0 msgr2=0x55b679df8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df3da0 0x55b679df8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 -- 172.21.15.64:0/2807521133 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a4072fa0 con 0x55b679deb2d0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 0x55b679df3860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f059c015a50 tx=0x7f059c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb631c96825d608 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f0582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.554+0000 7f0582ffd700 1 -- 172.21.15.64:0/2807521133 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c0137d0 con 0x55b679deb2d0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.555+0000 7f0582ffd700 1 -- 172.21.15.64:0/2807521133 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f059c013930 con 0x55b679deb2d0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.555+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b679c93210 con 0x55b679deb2d0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.555+0000 7f0582ffd700 1 -- 172.21.15.64:0/2807521133 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f059c013af0 con 0x55b679deb2d0 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.555+0000 7f05b61e1b80 10 client.25213 fetch_fsmap learned FSMap version 36 2024-05-08T21:58:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.555+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b679b67b00 con 0x55b679deb2d0 2024-05-08T21:58:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.555+0000 7f0582ffd700 1 -- 172.21.15.64:0/2807521133 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b679c93210 con 0x55b679deb2d0 2024-05-08T21:58:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f0582ffd700 1 -- 172.21.15.64:0/2807521133 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f059c0273a0 con 0x55b679deb2d0 2024-05-08T21:58:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05b61e1b80 10 client.25213 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T21:58:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05b61e1b80 10 client.25213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05b61e1b80 20 client.25213 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679dfff30 0x55b679e02350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05b61e1b80 4 client.25213 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b679db97b0 con 0x55b679dfff30 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679dfff30 0x55b679e02350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f0582ffd700 10 client.25213 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.556+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679dfff30 0x55b679e02350 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff6af557907d1a7 server_cookie=56a57cb6baa91160 in_seq=0 out_seq=0 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f0582ffd700 1 -- 172.21.15.64:0/2807521133 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b679db97b0 con 0x55b679dfff30 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f0582ffd700 10 client.25213 handle_command_reply: tid=0 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 client.25213 shutdown 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 msgr2=0x55b679df3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 0x55b679df3860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f059c015a50 tx=0x7f059c004b40 comp rx=0 tx=0).stop 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 shutdown_connections 2024-05-08T21:58:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df4220 0x55b679dfb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679dfff30 0x55b679e02350 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679deb2d0 0x55b679df3860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2807521133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df3da0 0x55b679df8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 >> 172.21.15.64:0/2807521133 conn(0x55b679d481b0 msgr2=0x55b679de4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.557+0000 7f0582ffd700 0 client.25213 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.558+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 shutdown_connections 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.558+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2807521133 wait complete. 2024-05-08T21:58:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.558+0000 7f05b61e1b80 20 client.25213 trim_cache size 0 max 16384 2024-05-08T21:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 Processor -- start 2024-05-08T21:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 -- start start 2024-05-08T21:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679deb2d0 0x55b679dfbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 0x55b679e06cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df4220 0x55b679df8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679deb190 con 0x55b679df4220 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679de5960 con 0x55b679deb2d0 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.585+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679df47b0 con 0x55b679df3da0 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 0x55b679e06cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 0x55b679e06cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56090/0 (socket says 172.21.15.64:56090) 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2105806324 learned_addr learned my addr 172.21.15.64:0/2105806324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a3fff700 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df4220 0x55b679df8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679deb2d0 0x55b679dfbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2105806324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679deb2d0 msgr2=0x55b679dfbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679deb2d0 0x55b679dfbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2105806324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df4220 msgr2=0x55b679df8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df4220 0x55b679df8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.586+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2105806324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b679d00500 con 0x55b679df3da0 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.588+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 0x55b679e06cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b679dea820 tx=0x7f059400b340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150be3bcaaf9a13a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.588+0000 7f058a7fc700 1 -- 172.21.15.64:0/2105806324 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594019de0 con 0x55b679df3da0 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.588+0000 7f058a7fc700 1 -- 172.21.15.64:0/2105806324 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05940198a0 con 0x55b679df3da0 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.588+0000 7f058a7fc700 1 -- 172.21.15.64:0/2105806324 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011440 con 0x55b679df3da0 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2105806324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 msgr2=0x55b679e06cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 0x55b679e06cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b679dea820 tx=0x7f059400b340 comp rx=0 tx=0).stop 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2105806324 shutdown_connections 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679deb2d0 0x55b679dfbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679df4220 0x55b679df8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2105806324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df3da0 0x55b679e06cb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2105806324 >> 172.21.15.64:0/2105806324 conn(0x55b679d481b0 msgr2=0x55b679df3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2105806324 shutdown_connections 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.589+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2105806324 wait complete. 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 -- start start 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 Processor -- start 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679df9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 0x55b679e52400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df4220 0x55b679e54950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679e55010 con 0x55b679deb2d0 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679e55180 con 0x55b679df3da0 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679e55340 con 0x55b679df4220 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.590+0000 7f05b61e1b80 10 client.0 _ll_register_callbacks cb 0x55b679cdfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df4220 0x55b679e54950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 0x55b679e52400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 0x55b679e52400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54352/0 (socket says 172.21.15.64:54352) 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2894558900 learned_addr learned my addr 172.21.15.64:0/2894558900 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679df9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2894558900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df4220 msgr2=0x55b679e54950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df4220 0x55b679e54950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2894558900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 msgr2=0x55b679df9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679df9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.591+0000 7f05a2ffd700 1 -- 172.21.15.64:0/2894558900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0594003680 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f05a2ffd700 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 0x55b679e52400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f059400bbd0 tx=0x7f059400be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e2797ccba58311 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f0582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f0582ffd700 1 -- 172.21.15.64:0/2894558900 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400b6c0 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05980cb160 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f0582ffd700 1 -- 172.21.15.64:0/2894558900 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0594016070 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f0582ffd700 1 -- 172.21.15.64:0/2894558900 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05940193e0 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f05b61e1b80 10 client.25295 fetch_fsmap learned FSMap version 36 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b679bbfc70 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.592+0000 7f0582ffd700 1 -- 172.21.15.64:0/2894558900 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05980cb160 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f0582ffd700 1 -- 172.21.15.64:0/2894558900 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0594022c70 con 0x55b679df3da0 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05b61e1b80 10 client.25295 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05b61e1b80 10 client.25295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05b61e1b80 20 client.25295 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679e56d60 0x55b679e59180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05b61e1b80 4 client.25295 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b679d32e60 con 0x55b679e56d60 2024-05-08T21:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679e56d60 0x55b679e59180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f05a37fe700 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679e56d60 0x55b679e59180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1a0c6873e37b5f server_cookie=ebca77eec3e97a70 in_seq=0 out_seq=0 2024-05-08T21:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.593+0000 7f0582ffd700 10 client.25295 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.647+0000 7f0582ffd700 1 -- 172.21.15.64:0/2894558900 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b679d32e60 con 0x55b679e56d60 2024-05-08T21:58:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.647+0000 7f0582ffd700 10 client.25295 handle_command_reply: tid=0 2024-05-08T21:58:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.647+0000 7f05b61e1b80 1 client.25295 shutdown 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 msgr2=0x55b679e52400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 0x55b679e52400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f059400bbd0 tx=0x7f059400be40 comp rx=0 tx=0).stop 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 shutdown_connections 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b679e56d60 0x55b679e59180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b679df4220 0x55b679e54950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b679deb2d0 0x55b679df9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f0582ffd700 0 client.25295 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 --2- 172.21.15.64:0/2894558900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b679df3da0 0x55b679e52400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 >> 172.21.15.64:0/2894558900 conn(0x55b679d481b0 msgr2=0x55b679e06cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 shutdown_connections 2024-05-08T21:58:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.648+0000 7f05b61e1b80 1 -- 172.21.15.64:0/2894558900 wait complete. 2024-05-08T21:58:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.650+0000 7f05b61e1b80 20 client.25295 trim_cache size 0 max 16384 2024-05-08T21:58:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:58:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:58:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:58:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f058c0a1320 msgr2=0x7f058c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f058c0a1320 0x7f058c0a37e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05a4108590 tx=0x7f059c00b040 comp rx=0 tx=0).stop 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 msgr2=0x7f05a4072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 0x7f05a4072020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0598009230 tx=0x7f059800b840 comp rx=0 tx=0).stop 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 shutdown_connections 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f058c0a1320 0x7f058c0a37e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4107540 0x7f05a4070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a41090d0 0x7f05a4072020 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 --2- 172.21.15.64:0/3802280058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4108730 0x7f05a4071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 >> 172.21.15.64:0/3802280058 conn(0x7f05a40764d0 msgr2=0x7f05a410ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 shutdown_connections 2024-05-08T21:58:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:23.651+0000 7f05aa027700 1 -- 172.21.15.64:0/3802280058 wait complete. 2024-05-08T21:58:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:23 smithi160 ceph-mon[46962]: pgmap v372: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 765 op/s 2024-05-08T21:58:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:23 smithi186 ceph-mon[46619]: pgmap v372: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 765 op/s 2024-05-08T21:58:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:23 smithi064 ceph-mon[42500]: pgmap v372: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 765 op/s 2024-05-08T21:58:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:25 smithi160 ceph-mon[46962]: pgmap v373: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 845 op/s 2024-05-08T21:58:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:25 smithi186 ceph-mon[46619]: pgmap v373: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 845 op/s 2024-05-08T21:58:26.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:25 smithi064 ceph-mon[42500]: pgmap v373: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 845 op/s 2024-05-08T21:58:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 215.0 s, 2570.7 tps, lat 39.032 ms stddev 42.129 2024-05-08T21:58:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:27 smithi160 ceph-mon[46962]: pgmap v374: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 794 op/s 2024-05-08T21:58:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:27 smithi186 ceph-mon[46619]: pgmap v374: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 794 op/s 2024-05-08T21:58:28.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:27 smithi064 ceph-mon[42500]: pgmap v374: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 794 op/s 2024-05-08T21:58:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:28 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:29.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:58:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:28 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:29.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:28 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:29.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:58:29.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:29.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:28 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:29.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:28 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:29.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:58:29.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:29.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:28 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:30.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:29 smithi186 ceph-mon[46619]: pgmap v375: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 831 op/s 2024-05-08T21:58:30.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:58:30.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:58:30.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:29 smithi064 ceph-mon[42500]: pgmap v375: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 831 op/s 2024-05-08T21:58:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:58:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:58:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:30.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:29 smithi160 ceph-mon[46962]: pgmap v375: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 831 op/s 2024-05-08T21:58:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:58:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:58:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:58:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 220.0 s, 2708.4 tps, lat 36.847 ms stddev 41.871 2024-05-08T21:58:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:31 smithi186 ceph-mon[46619]: pgmap v376: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 921 op/s 2024-05-08T21:58:32.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:31 smithi064 ceph-mon[42500]: pgmap v376: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 921 op/s 2024-05-08T21:58:32.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:31 smithi160 ceph-mon[46962]: pgmap v376: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 921 op/s 2024-05-08T21:58:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:33 smithi186 ceph-mon[46619]: pgmap v377: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 917 op/s 2024-05-08T21:58:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:33 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:34.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:33 smithi064 ceph-mon[42500]: pgmap v377: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 917 op/s 2024-05-08T21:58:34.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:33 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:34.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:33 smithi160 ceph-mon[46962]: pgmap v377: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 917 op/s 2024-05-08T21:58:34.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:33 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:35 smithi186 ceph-mon[46619]: pgmap v378: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.41k op/s 2024-05-08T21:58:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:35 smithi064 ceph-mon[42500]: pgmap v378: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.41k op/s 2024-05-08T21:58:36.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:35 smithi160 ceph-mon[46962]: pgmap v378: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.41k op/s 2024-05-08T21:58:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 225.0 s, 1735.8 tps, lat 44.924 ms stddev 84.132 2024-05-08T21:58:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.772+0000 7fb85716b700 1 -- 172.21.15.64:0/1721774181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850102a30 msgr2=0x7fb850109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- 172.21.15.64:0/1721774181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850102a30 0x7fb850109350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb848009f90 tx=0x7fb84800b330 comp rx=0 tx=0).stop 2024-05-08T21:58:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- 172.21.15.64:0/1721774181 shutdown_connections 2024-05-08T21:58:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- 172.21.15.64:0/1721774181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850102a30 0x7fb850109350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- 172.21.15.64:0/1721774181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850100f20 0x7fb850101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- 172.21.15.64:0/1721774181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102070 0x7fb8501024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- 172.21.15.64:0/1721774181 >> 172.21.15.64:0/1721774181 conn(0x7fb8500fc490 msgr2=0x7fb8500fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- 172.21.15.64:0/1721774181 shutdown_connections 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- 172.21.15.64:0/1721774181 wait complete. 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 Processor -- start 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- start start 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 0x7fb850074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102070 0x7fb850073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102a30 0x7fb8500738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb850074070 con 0x7fb850102a30 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500741e0 con 0x7fb850100f20 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.773+0000 7fb85716b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8501a0fe0 con 0x7fb850102070 2024-05-08T21:58:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 0x7fb850074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb855708700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102a30 0x7fb8500738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 0x7fb850074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:58488/0 (socket says 172.21.15.64:58488) 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 -- 172.21.15.64:0/1422421862 learned_addr learned my addr 172.21.15.64:0/1422421862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb84ffff700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102070 0x7fb850073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 -- 172.21.15.64:0/1422421862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102070 msgr2=0x7fb850073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102070 0x7fb850073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 -- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102a30 msgr2=0x7fb8500738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102a30 0x7fb8500738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 -- 172.21.15.64:0/1422421862 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c0095f0 con 0x7fb850100f20 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb855708700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102a30 0x7fb8500738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb854f07700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 0x7fb850074d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb84400e9a0 tx=0x7fb84400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb84dffb700 1 -- 172.21.15.64:0/1422421862 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844009790 con 0x7fb850100f20 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb84dffb700 1 -- 172.21.15.64:0/1422421862 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84400cb70 con 0x7fb850100f20 2024-05-08T21:58:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb848009c40 con 0x7fb850100f20 2024-05-08T21:58:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb84dffb700 1 -- 172.21.15.64:0/1422421862 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844010ea0 con 0x7fb850100f20 2024-05-08T21:58:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.774+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8501a15f0 con 0x7fb850100f20 2024-05-08T21:58:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.777+0000 7fb84dffb700 1 -- 172.21.15.64:0/1422421862 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb84400cce0 con 0x7fb850100f20 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.777+0000 7fb84dffb700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8400a1120 0x7fb8400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb84ffff700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8400a1120 0x7fb8400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb84dffb700 1 -- 172.21.15.64:0/1422421862 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb844014070 con 0x7fb850100f20 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb84ffff700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8400a1120 0x7fb8400a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fd0 tx=0x7fb83c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 Processor -- start 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 -- start start 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511271ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112723e0 0x55d511272cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511273750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5111c90f0 con 0x55d511271a80 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5111c6ab0 con 0x55d5112732d0 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.778+0000 7fb863325b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5111c23a0 con 0x55d5112723e0 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511271ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb84ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112723e0 0x55d511272cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511271ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45716/0 (socket says 172.21.15.64:45716) 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 -- 172.21.15.64:0/3686399691 learned_addr learned my addr 172.21.15.64:0/3686399691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb855708700 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511273750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 -- 172.21.15.64:0/3686399691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112723e0 msgr2=0x55d511272cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112723e0 0x55d511272cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 -- 172.21.15.64:0/3686399691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 msgr2=0x55d511273750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511273750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 -- 172.21.15.64:0/3686399691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d511043350 con 0x55d511271a80 2024-05-08T21:58:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb84ffff700 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112723e0 0x55d511272cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb855708700 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511273750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb854f07700 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511271ea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb8440a1ee0 tx=0x7fb8440998f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4e900056683f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb83affd700 1 -- 172.21.15.64:0/3686399691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844099f90 con 0x55d511271a80 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb83affd700 1 -- 172.21.15.64:0/3686399691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84409ddc0 con 0x55d511271a80 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.779+0000 7fb83affd700 1 -- 172.21.15.64:0/3686399691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440a21e0 con 0x55d511271a80 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 -- 172.21.15.64:0/3686399691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 msgr2=0x55d511271ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511271ea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb8440a1ee0 tx=0x7fb8440998f0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 -- 172.21.15.64:0/3686399691 shutdown_connections 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511273750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511271ea0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 --2- 172.21.15.64:0/3686399691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112723e0 0x55d511272cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 -- 172.21.15.64:0/3686399691 >> 172.21.15.64:0/3686399691 conn(0x55d5111d01b0 msgr2=0x55d51126e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 -- 172.21.15.64:0/3686399691 shutdown_connections 2024-05-08T21:58:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 -- 172.21.15.64:0/3686399691 wait complete. 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 -- start start 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.780+0000 7fb863325b80 1 Processor -- start 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d51127b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d51127bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511282de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d51115e500 con 0x55d5112723e0 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d51115e1a0 con 0x55d5112732d0 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d511045640 con 0x55d511271a80 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb863325b80 10 client.0 _ll_register_callbacks cb 0x55d511167ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d51127b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d51127b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:59784/0 (socket says 172.21.15.64:59784) 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 -- 172.21.15.64:0/1983514900 learned_addr learned my addr 172.21.15.64:0/1983514900 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb84ffff700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d51127bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb855708700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511282de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 -- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 msgr2=0x55d511282de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511282de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 -- 172.21.15.64:0/1983514900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 msgr2=0x55d51127bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d51127bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 -- 172.21.15.64:0/1983514900 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb850109c80 con 0x55d511271a80 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb854f07700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d51127b860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb844096490 tx=0x7fb8440997b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41327462aa255572 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.781+0000 7fb82ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb82ffff700 1 -- 172.21.15.64:0/1983514900 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440b8320 con 0x55d511271a80 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb82ffff700 1 -- 172.21.15.64:0/1983514900 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8440a2410 con 0x55d511271a80 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d51111b210 con 0x55d511271a80 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb82ffff700 1 -- 172.21.15.64:0/1983514900 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb844099f90 con 0x55d511271a80 2024-05-08T21:58:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 10 client.25231 fetch_fsmap learned FSMap version 38 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fb8501a15f0 con 0x55d511271a80 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb82ffff700 1 -- 172.21.15.64:0/1983514900 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d51111b210 con 0x55d511271a80 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb82ffff700 1 -- 172.21.15.64:0/1983514900 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb844095ea0 con 0x55d511271a80 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 10 client.25231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 20 client.25231 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d511287330 0x55d511289750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 4 client.25231 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5112417b0 con 0x55d511287330 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.782+0000 7fb84ffff700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d511287330 0x55d511289750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb84ffff700 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d511287330 0x55d511289750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=456a4403f18702f5 server_cookie=fbe37b27b7cae613 in_seq=0 out_seq=0 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb82ffff700 10 client.25231 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb82ffff700 1 -- 172.21.15.64:0/1983514900 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5112417b0 con 0x55d511287330 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb82ffff700 10 client.25231 handle_command_reply: tid=0 2024-05-08T21:58:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 client.25231 shutdown 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 msgr2=0x55d51127b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d51127b860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb844096490 tx=0x7fb8440997b0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 shutdown_connections 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d511282de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d511287330 0x55d511289750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d51127b860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb863325b80 1 --2- 172.21.15.64:0/1983514900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d51127bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.783+0000 7fb82ffff700 0 client.25231 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.784+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 >> 172.21.15.64:0/1983514900 conn(0x55d5111d01b0 msgr2=0x55d51126c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.784+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 shutdown_connections 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.784+0000 7fb863325b80 1 -- 172.21.15.64:0/1983514900 wait complete. 2024-05-08T21:58:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.784+0000 7fb863325b80 20 client.25231 trim_cache size 0 max 16384 2024-05-08T21:58:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 Processor -- start 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 -- start start 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511289720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112723e0 0x55d511271e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112732d0 0x55d511283490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d51126d930 con 0x55d511271a80 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d511279530 con 0x55d5112723e0 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb863325b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d511276e20 con 0x55d5112732d0 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb854f07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511289720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb854f07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511289720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45726/0 (socket says 172.21.15.64:45726) 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb854f07700 1 -- 172.21.15.64:0/3300888291 learned_addr learned my addr 172.21.15.64:0/3300888291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb855708700 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112732d0 0x55d511283490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.799+0000 7fb84ffff700 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112723e0 0x55d511271e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb855708700 1 -- 172.21.15.64:0/3300888291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112723e0 msgr2=0x55d511271e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb855708700 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112723e0 0x55d511271e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb855708700 1 -- 172.21.15.64:0/3300888291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 msgr2=0x55d511289720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb855708700 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511289720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb855708700 1 -- 172.21.15.64:0/3300888291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d510fefb00 con 0x55d5112732d0 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb84ffff700 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112723e0 0x55d511271e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb855708700 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112732d0 0x55d511283490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb848007a60 tx=0x7fb848007d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf27336650bf8a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb83b7fe700 1 -- 172.21.15.64:0/3300888291 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800e040 con 0x55d5112732d0 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb83b7fe700 1 -- 172.21.15.64:0/3300888291 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84800be50 con 0x55d5112732d0 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.800+0000 7fb83b7fe700 1 -- 172.21.15.64:0/3300888291 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480056a0 con 0x55d5112732d0 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 -- 172.21.15.64:0/3300888291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112732d0 msgr2=0x55d511283490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112732d0 0x55d511283490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb848007a60 tx=0x7fb848007d80 comp rx=0 tx=0).stop 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 -- 172.21.15.64:0/3300888291 shutdown_connections 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5112732d0 0x55d511283490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d511271a80 0x55d511289720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 --2- 172.21.15.64:0/3300888291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112723e0 0x55d511271e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 -- 172.21.15.64:0/3300888291 >> 172.21.15.64:0/3300888291 conn(0x55d5111d01b0 msgr2=0x55d51127ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 -- 172.21.15.64:0/3300888291 shutdown_connections 2024-05-08T21:58:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 -- 172.21.15.64:0/3300888291 wait complete. 2024-05-08T21:58:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 -- start start 2024-05-08T21:58:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.801+0000 7fb863325b80 1 Processor -- start 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d511280590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d511280ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d5112819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d511281ee0 con 0x55d5112723e0 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d511282050 con 0x55d5112732d0 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d511282210 con 0x55d511271a80 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb863325b80 10 client.0 _ll_register_callbacks cb 0x55d511167ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb84ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d511280ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d511280590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d511280590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:59798/0 (socket says 172.21.15.64:59798) 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 -- 172.21.15.64:0/2972228260 learned_addr learned my addr 172.21.15.64:0/2972228260 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb84ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d511280ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45742/0 (socket says 172.21.15.64:45742) 2024-05-08T21:58:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb855708700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d5112819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 -- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 msgr2=0x55d5112819a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d5112819a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 -- 172.21.15.64:0/2972228260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 msgr2=0x55d511280ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d511280ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 -- 172.21.15.64:0/2972228260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c011d90 con 0x55d511271a80 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb84ffff700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d511280ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb854f07700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d511280590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb844018720 tx=0x7fb8440998f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e422ddb5c99571c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb82ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.802+0000 7fb82ffff700 1 -- 172.21.15.64:0/2972228260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844099640 con 0x55d511271a80 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb84409ddc0 con 0x55d511271a80 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb82ffff700 1 -- 172.21.15.64:0/2972228260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8440a2410 con 0x55d511271a80 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb82ffff700 1 -- 172.21.15.64:0/2972228260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb844099f90 con 0x55d511271a80 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 10 client.25237 fetch_fsmap learned FSMap version 38 2024-05-08T21:58:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fb848007b40 con 0x55d511271a80 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb82ffff700 1 -- 172.21.15.64:0/2972228260 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84409ddc0 con 0x55d511271a80 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb82ffff700 1 -- 172.21.15.64:0/2972228260 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb844099100 con 0x55d511271a80 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 10 client.25237 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 10 client.25237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 20 client.25237 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d5112ddb90 0x55d5112dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 4 client.25237 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5111bae60 con 0x55d5112ddb90 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.803+0000 7fb84ffff700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d5112ddb90 0x55d5112dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.804+0000 7fb84ffff700 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d5112ddb90 0x55d5112dffb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac218df0ed6b127 server_cookie=b6708b4e53d43885 in_seq=0 out_seq=0 2024-05-08T21:58:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.804+0000 7fb82ffff700 10 client.25237 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.816+0000 7fb82ffff700 1 -- 172.21.15.64:0/2972228260 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5111bae60 con 0x55d5112ddb90 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.816+0000 7fb82ffff700 10 client.25237 handle_command_reply: tid=0 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.816+0000 7fb863325b80 1 client.25237 shutdown 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 msgr2=0x55d511280590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d511280590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb844018720 tx=0x7fb8440998f0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 shutdown_connections 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d5112732d0 0x55d5112819a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d5112ddb90 0x55d5112dffb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d511271a80 0x55d511280590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 --2- 172.21.15.64:0/2972228260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5112723e0 0x55d511280ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb82ffff700 0 client.25237 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 >> 172.21.15.64:0/2972228260 conn(0x55d5111d01b0 msgr2=0x55d511279850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 shutdown_connections 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 1 -- 172.21.15.64:0/2972228260 wait complete. 2024-05-08T21:58:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb863325b80 20 client.25237 trim_cache size 0 max 16384 2024-05-08T21:58:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:58:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:58:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:58:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8400a1120 msgr2=0x7fb8400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8400a1120 0x7fb8400a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fd0 tx=0x7fb83c009310 comp rx=0 tx=0).stop 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 msgr2=0x7fb850074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 0x7fb850074d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb84400e9a0 tx=0x7fb84400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 shutdown_connections 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8400a1120 0x7fb8400a35e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb850100f20 0x7fb850074d00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb850102a30 0x7fb8500738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.817+0000 7fb85716b700 1 --2- 172.21.15.64:0/1422421862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102070 0x7fb850073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.818+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 >> 172.21.15.64:0/1422421862 conn(0x7fb8500fc490 msgr2=0x7fb8501054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.818+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 shutdown_connections 2024-05-08T21:58:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:36.818+0000 7fb85716b700 1 -- 172.21.15.64:0/1422421862 wait complete. 2024-05-08T21:58:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:37 smithi064 ceph-mon[42500]: pgmap v379: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 1.59k op/s 2024-05-08T21:58:38.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:37 smithi160 ceph-mon[46962]: pgmap v379: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 1.59k op/s 2024-05-08T21:58:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:37 smithi186 ceph-mon[46619]: pgmap v379: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 1.59k op/s 2024-05-08T21:58:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:39 smithi186 ceph-mon[46619]: pgmap v380: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 MiB/s wr, 2.03k op/s 2024-05-08T21:58:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:39 smithi064 ceph-mon[42500]: pgmap v380: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 MiB/s wr, 2.03k op/s 2024-05-08T21:58:40.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:39 smithi160 ceph-mon[46962]: pgmap v380: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 MiB/s wr, 2.03k op/s 2024-05-08T21:58:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 230.0 s, 2386.4 tps, lat 51.092 ms stddev 174.438 2024-05-08T21:58:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:41 smithi064 ceph-mon[42500]: pgmap v381: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 MiB/s wr, 2.04k op/s 2024-05-08T21:58:42.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:41 smithi160 ceph-mon[46962]: pgmap v381: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 MiB/s wr, 2.04k op/s 2024-05-08T21:58:42.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:41 smithi186 ceph-mon[46619]: pgmap v381: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 MiB/s wr, 2.04k op/s 2024-05-08T21:58:44.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:43 smithi064 ceph-mon[42500]: pgmap v382: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.96k op/s 2024-05-08T21:58:44.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:43 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:44.299 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:43 smithi160 ceph-mon[46962]: pgmap v382: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.96k op/s 2024-05-08T21:58:44.299 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:43 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:44.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:43 smithi186 ceph-mon[46619]: pgmap v382: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.96k op/s 2024-05-08T21:58:44.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:43 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:58:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:45 smithi064 ceph-mon[42500]: pgmap v383: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 MiB/s wr, 1.93k op/s 2024-05-08T21:58:46.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:45 smithi160 ceph-mon[46962]: pgmap v383: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 MiB/s wr, 1.93k op/s 2024-05-08T21:58:46.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:45 smithi186 ceph-mon[46619]: pgmap v383: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 MiB/s wr, 1.93k op/s 2024-05-08T21:58:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 235.0 s, 2840.1 tps, lat 35.429 ms stddev 42.003 2024-05-08T21:58:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:47 smithi064 ceph-mon[42500]: pgmap v384: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 MiB/s wr, 1.35k op/s 2024-05-08T21:58:48.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:47 smithi160 ceph-mon[46962]: pgmap v384: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 MiB/s wr, 1.35k op/s 2024-05-08T21:58:48.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:47 smithi186 ceph-mon[46619]: pgmap v384: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 MiB/s wr, 1.35k op/s 2024-05-08T21:58:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 -- 172.21.15.64:0/810115414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830103140 msgr2=0x7f4830103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 --2- 172.21.15.64:0/810115414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830103140 0x7f4830103560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48200099c0 tx=0x7f4820009cd0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 -- 172.21.15.64:0/810115414 shutdown_connections 2024-05-08T21:58:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 --2- 172.21.15.64:0/810115414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830103140 0x7f4830103560 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 --2- 172.21.15.64:0/810115414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830104cd0 0x7f483010b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 --2- 172.21.15.64:0/810115414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830104330 0x7f4830104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 -- 172.21.15.64:0/810115414 >> 172.21.15.64:0/810115414 conn(0x7f48300fe6b0 msgr2=0x7f4830100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.998+0000 7f4834df9700 1 -- 172.21.15.64:0/810115414 shutdown_connections 2024-05-08T21:58:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.999+0000 7f4834df9700 1 -- 172.21.15.64:0/810115414 wait complete. 2024-05-08T21:58:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:49.999+0000 7f4834df9700 1 Processor -- start 2024-05-08T21:58:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 -- start start 2024-05-08T21:58:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103140 0x7f483006cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 0x7f483006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830104cd0 0x7f4830072450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830072a20 con 0x7f4830103140 2024-05-08T21:58:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830072b60 con 0x7f4830104330 2024-05-08T21:58:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f4834df9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830072cf0 con 0x7f4830104cd0 2024-05-08T21:58:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f482dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 0x7f483006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f482dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 0x7f483006d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45180/0 (socket says 172.21.15.64:45180) 2024-05-08T21:58:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f482dd9b700 1 -- 172.21.15.64:0/3982679173 learned_addr learned my addr 172.21.15.64:0/3982679173 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f482ed9d700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830104cd0 0x7f4830072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.000+0000 7f482e59c700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103140 0x7f483006cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f482dd9b700 1 -- 172.21.15.64:0/3982679173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830104cd0 msgr2=0x7f4830072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830104cd0 0x7f4830072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f482dd9b700 1 -- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103140 msgr2=0x7f483006cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103140 0x7f483006cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f482dd9b700 1 -- 172.21.15.64:0/3982679173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200095f0 con 0x7f4830104330 2024-05-08T21:58:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 0x7f483006d440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9a0 tx=0x7f482400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f481f7fe700 1 -- 172.21.15.64:0/3982679173 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824009790 con 0x7f4830104330 2024-05-08T21:58:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.001+0000 7f481f7fe700 1 -- 172.21.15.64:0/3982679173 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f482400cbd0 con 0x7f4830104330 2024-05-08T21:58:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.002+0000 7f481f7fe700 1 -- 172.21.15.64:0/3982679173 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824018680 con 0x7f4830104330 2024-05-08T21:58:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.002+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4830072fd0 con 0x7f4830104330 2024-05-08T21:58:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.002+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483010dd40 con 0x7f4830104330 2024-05-08T21:58:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.005+0000 7f481f7fe700 1 -- 172.21.15.64:0/3982679173 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f482400cd40 con 0x7f4830104330 2024-05-08T21:58:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.006+0000 7f481f7fe700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48180a1270 0x7f48180a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.006+0000 7f4840fb3b80 1 Processor -- start 2024-05-08T21:58:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.006+0000 7f481f7fe700 1 -- 172.21.15.64:0/3982679173 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4824014070 con 0x7f4830104330 2024-05-08T21:58:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f482e59c700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48180a1270 0x7f48180a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 -- start start 2024-05-08T21:58:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 0x55c6a0178ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0179cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a017a2d0 0x55c6a017a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a00d00f0 con 0x55c6a0178a80 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a00cdab0 con 0x55c6a01793e0 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a00c93a0 con 0x55c6a017a2d0 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f482e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 0x55c6a0178ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f482e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 0x55c6a0178ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53760/0 (socket says 172.21.15.64:53760) 2024-05-08T21:58:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f482e59c700 1 -- 172.21.15.64:0/4230456696 learned_addr learned my addr 172.21.15.64:0/4230456696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f482dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0179cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.007+0000 7f482ed9d700 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a017a2d0 0x55c6a017a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482e59c700 1 -- 172.21.15.64:0/4230456696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a017a2d0 msgr2=0x55c6a017a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482e59c700 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a017a2d0 0x55c6a017a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482e59c700 1 -- 172.21.15.64:0/4230456696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 msgr2=0x55c6a0179cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482e59c700 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0179cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482e59c700 1 -- 172.21.15.64:0/4230456696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c69ff4a350 con 0x55c6a0178a80 2024-05-08T21:58:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482e59c700 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 0x55c6a0178ea0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4820006010 tx=0x7f482000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4118467bbd16db91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.008+0000 7f482ed9d700 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a017a2d0 0x55c6a017a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:58:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.009+0000 7f481cff9700 1 -- 172.21.15.64:0/4230456696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820004e40 con 0x55c6a0178a80 2024-05-08T21:58:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.009+0000 7f481cff9700 1 -- 172.21.15.64:0/4230456696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f482000f460 con 0x55c6a0178a80 2024-05-08T21:58:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.009+0000 7f481cff9700 1 -- 172.21.15.64:0/4230456696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48200316b0 con 0x55c6a0178a80 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4230456696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 msgr2=0x55c6a0178ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f482e59c700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48180a1270 0x7f48180a3730 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f48200099c0 tx=0x7f482001c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 0x55c6a0178ea0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4820006010 tx=0x7f482000bdb0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4230456696 shutdown_connections 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a017a2d0 0x55c6a017a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a0178a80 0x55c6a0178ea0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4230456696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0179cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4230456696 >> 172.21.15.64:0/4230456696 conn(0x55c6a00d71b0 msgr2=0x55c6a0175440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4230456696 shutdown_connections 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.010+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4230456696 wait complete. 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 -- start start 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 Processor -- start 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0182790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a0182cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0065500 con 0x55c6a01793e0 2024-05-08T21:58:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a00651a0 con 0x55c6a017a2d0 2024-05-08T21:58:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69ff4c640 con 0x55c6a0178a80 2024-05-08T21:58:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f4840fb3b80 10 client.0 _ll_register_callbacks cb 0x55c6a006ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f482ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f482ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45190/0 (socket says 172.21.15.64:45190) 2024-05-08T21:58:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f482ed9d700 1 -- 172.21.15.64:0/3272718868 learned_addr learned my addr 172.21.15.64:0/3272718868 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a0182cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.011+0000 7f482e59c700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0182790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f482ed9d700 1 -- 172.21.15.64:0/3272718868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 msgr2=0x55c6a0182790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f482ed9d700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0182790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f482ed9d700 1 -- 172.21.15.64:0/3272718868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 msgr2=0x55c6a0182cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f482ed9d700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a0182cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f482ed9d700 1 -- 172.21.15.64:0/3272718868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483010bed0 con 0x55c6a017a2d0 2024-05-08T21:58:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f482ed9d700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4828007a80 tx=0x7f4828007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a4299740c00b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f4815ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f4815ffb700 1 -- 172.21.15.64:0/3272718868 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280118d0 con 0x55c6a017a2d0 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f4815ffb700 1 -- 172.21.15.64:0/3272718868 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4828011a30 con 0x55c6a017a2d0 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a0022210 con 0x55c6a017a2d0 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.012+0000 7f4815ffb700 1 -- 172.21.15.64:0/3272718868 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4828011bf0 con 0x55c6a017a2d0 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.013+0000 7f4840fb3b80 10 client.25325 fetch_fsmap learned FSMap version 39 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.013+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c69fef6b00 con 0x55c6a017a2d0 2024-05-08T21:58:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.013+0000 7f4815ffb700 1 -- 172.21.15.64:0/3272718868 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6a0022210 con 0x55c6a017a2d0 2024-05-08T21:58:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4815ffb700 1 -- 172.21.15.64:0/3272718868 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4828014070 con 0x55c6a017a2d0 2024-05-08T21:58:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4840fb3b80 10 client.25325 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:58:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4840fb3b80 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4840fb3b80 20 client.25325 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a018e5f0 0x55c6a0190a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4840fb3b80 4 client.25325 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:58:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a01487b0 con 0x55c6a018e5f0 2024-05-08T21:58:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.014+0000 7f482e59c700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a018e5f0 0x55c6a0190a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.015+0000 7f482e59c700 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a018e5f0 0x55c6a0190a10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62fbcd897722826 server_cookie=4c6f031f66b15a9b in_seq=0 out_seq=0 2024-05-08T21:58:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.015+0000 7f4815ffb700 10 client.25325 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4815ffb700 1 -- 172.21.15.64:0/3272718868 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6a01487b0 con 0x55c6a018e5f0 2024-05-08T21:58:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4815ffb700 10 client.25325 handle_command_reply: tid=0 2024-05-08T21:58:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 client.25325 shutdown 2024-05-08T21:58:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 msgr2=0x55c6a0189d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4828007a80 tx=0x7f4828007d90 comp rx=0 tx=0).stop 2024-05-08T21:58:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 shutdown_connections 2024-05-08T21:58:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189d10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a018e5f0 0x55c6a0190a10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0182790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3272718868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a0182cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 >> 172.21.15.64:0/3272718868 conn(0x55c6a00d71b0 msgr2=0x55c6a0173860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4815ffb700 0 client.25325 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 shutdown_connections 2024-05-08T21:58:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3272718868 wait complete. 2024-05-08T21:58:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.016+0000 7f4840fb3b80 20 client.25325 trim_cache size 0 max 16384 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 Processor -- start 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 -- start start 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0181ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a018a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a01487b0 con 0x55c6a01793e0 2024-05-08T21:58:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0022210 con 0x55c6a017a2d0 2024-05-08T21:58:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.040+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a016e570 con 0x55c6a0178a80 2024-05-08T21:58:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a018a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a018a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53772/0 (socket says 172.21.15.64:53772) 2024-05-08T21:58:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 -- 172.21.15.64:0/3048712027 learned_addr learned my addr 172.21.15.64:0/3048712027 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482e59c700 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0181ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 -- 172.21.15.64:0/3048712027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 msgr2=0x55c6a0181ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0181ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 -- 172.21.15.64:0/3048712027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 msgr2=0x55c6a0189cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.041+0000 7f482dd9b700 1 -- 172.21.15.64:0/3048712027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a008f500 con 0x55c6a01793e0 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.042+0000 7f482dd9b700 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a018a3c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f48240ca530 tx=0x7f4824010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da1049ab5e86eea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.042+0000 7f481d7fa700 1 -- 172.21.15.64:0/3048712027 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482409dcb0 con 0x55c6a01793e0 2024-05-08T21:58:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.042+0000 7f481d7fa700 1 -- 172.21.15.64:0/3048712027 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f482409ae90 con 0x55c6a01793e0 2024-05-08T21:58:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.042+0000 7f481d7fa700 1 -- 172.21.15.64:0/3048712027 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48240a1e70 con 0x55c6a01793e0 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3048712027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 msgr2=0x55c6a018a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a018a3c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f48240ca530 tx=0x7f4824010b50 comp rx=0 tx=0).stop 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3048712027 shutdown_connections 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a017a2d0 0x55c6a0189cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0181ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/3048712027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a01793e0 0x55c6a018a3c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3048712027 >> 172.21.15.64:0/3048712027 conn(0x55c6a00d71b0 msgr2=0x55c6a01808c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3048712027 shutdown_connections 2024-05-08T21:58:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.043+0000 7f4840fb3b80 1 -- 172.21.15.64:0/3048712027 wait complete. 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 -- start start 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 Processor -- start 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0186fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0187500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a017a2d0 0x55c6a0188510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0188a50 con 0x55c6a017a2d0 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0188bc0 con 0x55c6a01793e0 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0188d80 con 0x55c6a0178a80 2024-05-08T21:58:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.044+0000 7f4840fb3b80 10 client.0 _ll_register_callbacks cb 0x55c6a006ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.045+0000 7f482ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a017a2d0 0x55c6a0188510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.045+0000 7f482ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a017a2d0 0x55c6a0188510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53776/0 (socket says 172.21.15.64:53776) 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.045+0000 7f482ed9d700 1 -- 172.21.15.64:0/4048338721 learned_addr learned my addr 172.21.15.64:0/4048338721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482dd9b700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0187500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482e59c700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0186fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482dd9b700 1 -- 172.21.15.64:0/4048338721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 msgr2=0x55c6a0186fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482dd9b700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0186fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482dd9b700 1 -- 172.21.15.64:0/4048338721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a017a2d0 msgr2=0x55c6a0188510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482dd9b700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a017a2d0 0x55c6a0188510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482dd9b700 1 -- 172.21.15.64:0/4048338721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483010dd40 con 0x55c6a01793e0 2024-05-08T21:58:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.046+0000 7f482e59c700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0186fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f482dd9b700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0187500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f48240ca530 tx=0x7f4824099ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ecad4fe91ccfe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4815ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4815ffb700 1 -- 172.21.15.64:0/4048338721 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824095c70 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4815ffb700 1 -- 172.21.15.64:0/4048338721 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4824095dd0 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f482000f460 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4815ffb700 1 -- 172.21.15.64:0/4048338721 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48240b8250 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4840fb3b80 10 client.25331 fetch_fsmap learned FSMap version 39 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.047+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c69ff4ec70 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4815ffb700 1 -- 172.21.15.64:0/4048338721 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48240a8080 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4815ffb700 1 -- 172.21.15.64:0/4048338721 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f482409ebd0 con 0x55c6a01793e0 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4840fb3b80 10 client.25331 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4840fb3b80 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:58:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4840fb3b80 20 client.25331 populate_metadata read hostname 'smithi064' 2024-05-08T21:58:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a01e4ba0 0x55c6a01e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4840fb3b80 4 client.25331 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:58:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6a00c1e60 con 0x55c6a01e4ba0 2024-05-08T21:58:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.048+0000 7f482e59c700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a01e4ba0 0x55c6a01e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.049+0000 7f482e59c700 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a01e4ba0 0x55c6a01e6fc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ca3f8d06fba65dd server_cookie=a6e799a7429597aa in_seq=0 out_seq=0 2024-05-08T21:58:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.049+0000 7f4815ffb700 10 client.25331 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4815ffb700 1 -- 172.21.15.64:0/4048338721 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6a00c1e60 con 0x55c6a01e4ba0 2024-05-08T21:58:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4815ffb700 10 client.25331 handle_command_reply: tid=0 2024-05-08T21:58:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 client.25331 shutdown 2024-05-08T21:58:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 msgr2=0x55c6a0187500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0187500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f48240ca530 tx=0x7f4824099ff0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 shutdown_connections 2024-05-08T21:58:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55c6a01e4ba0 0x55c6a01e6fc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a017a2d0 0x55c6a0188510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6a0178a80 0x55c6a0186fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 --2- 172.21.15.64:0/4048338721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6a01793e0 0x55c6a0187500 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.062+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 >> 172.21.15.64:0/4048338721 conn(0x55c6a00d71b0 msgr2=0x55c6a0195370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.063+0000 7f4815ffb700 0 client.25331 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.063+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 shutdown_connections 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.063+0000 7f4840fb3b80 1 -- 172.21.15.64:0/4048338721 wait complete. 2024-05-08T21:58:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.063+0000 7f4840fb3b80 20 client.25331 trim_cache size 0 max 16384 2024-05-08T21:58:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:58:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:58:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:58:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:58:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48180a1270 msgr2=0x7f48180a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48180a1270 0x7f48180a3730 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f48200099c0 tx=0x7f482001c000 comp rx=0 tx=0).stop 2024-05-08T21:58:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 msgr2=0x7f483006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 0x7f483006d440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9a0 tx=0x7f482400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 shutdown_connections 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48180a1270 0x7f48180a3730 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830104cd0 0x7f4830072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103140 0x7f483006cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 --2- 172.21.15.64:0/3982679173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4830104330 0x7f483006d440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 >> 172.21.15.64:0/3982679173 conn(0x7f48300fe6b0 msgr2=0x7f4830079000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 shutdown_connections 2024-05-08T21:58:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:58:50.065+0000 7f4834df9700 1 -- 172.21.15.64:0/3982679173 wait complete. 2024-05-08T21:58:50.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:49 smithi160 ceph-mon[46962]: pgmap v385: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 24 MiB/s wr, 1.10k op/s 2024-05-08T21:58:50.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:49 smithi186 ceph-mon[46619]: pgmap v385: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 24 MiB/s wr, 1.10k op/s 2024-05-08T21:58:50.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:49 smithi064 ceph-mon[42500]: pgmap v385: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 24 MiB/s wr, 1.10k op/s 2024-05-08T21:58:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 240.0 s, 2981.2 tps, lat 33.360 ms stddev 36.818 2024-05-08T21:58:52.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:52 smithi160 ceph-mon[46962]: pgmap v386: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 745 op/s 2024-05-08T21:58:52.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:52 smithi186 ceph-mon[46619]: pgmap v386: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 745 op/s 2024-05-08T21:58:52.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:52 smithi064 ceph-mon[42500]: pgmap v386: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 745 op/s 2024-05-08T21:58:54.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:54 smithi160 ceph-mon[46962]: pgmap v387: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-08T21:58:54.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:54 smithi186 ceph-mon[46619]: pgmap v387: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-08T21:58:54.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:54 smithi064 ceph-mon[42500]: pgmap v387: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-08T21:58:56.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:56 smithi160 ceph-mon[46962]: pgmap v388: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-05-08T21:58:56.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:56 smithi186 ceph-mon[46619]: pgmap v388: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-05-08T21:58:56.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:56 smithi064 ceph-mon[42500]: pgmap v388: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-05-08T21:58:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 245.0 s, 2907.6 tps, lat 34.420 ms stddev 54.607 2024-05-08T21:58:58.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:58:58 smithi186 ceph-mon[46619]: pgmap v389: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 805 op/s 2024-05-08T21:58:58.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:58:58 smithi064 ceph-mon[42500]: pgmap v389: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 805 op/s 2024-05-08T21:58:58.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:58:58 smithi160 ceph-mon[46962]: pgmap v389: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 805 op/s 2024-05-08T21:59:00.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:00 smithi186 ceph-mon[46619]: pgmap v390: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 834 op/s 2024-05-08T21:59:00.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:00 smithi064 ceph-mon[42500]: pgmap v390: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 834 op/s 2024-05-08T21:59:00.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:00 smithi160 ceph-mon[46962]: pgmap v390: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 834 op/s 2024-05-08T21:59:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 250.0 s, 2147.8 tps, lat 46.542 ms stddev 52.023 2024-05-08T21:59:02.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:02 smithi186 ceph-mon[46619]: pgmap v391: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 838 op/s 2024-05-08T21:59:02.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:02 smithi064 ceph-mon[42500]: pgmap v391: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 838 op/s 2024-05-08T21:59:02.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:02 smithi160 ceph-mon[46962]: pgmap v391: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 838 op/s 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 -- 172.21.15.64:0/2982438729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 msgr2=0x7f5d800ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/2982438729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 0x7f5d800ff5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009990 tx=0x7f5d7c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 -- 172.21.15.64:0/2982438729 shutdown_connections 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/2982438729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 0x7f5d800ff5e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/2982438729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d801008e0 0x7f5d80105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/2982438729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d801003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 -- 172.21.15.64:0/2982438729 >> 172.21.15.64:0/2982438729 conn(0x7f5d800faaf0 msgr2=0x7f5d800fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 -- 172.21.15.64:0/2982438729 shutdown_connections 2024-05-08T21:59:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.242+0000 7f5d87f5e700 1 -- 172.21.15.64:0/2982438729 wait complete. 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 Processor -- start 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 -- start start 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 0x7f5d8006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d8006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d801008e0 0x7f5d801ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80072e80 con 0x7f5d800fff20 2024-05-08T21:59:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80072fc0 con 0x7f5d800ff1c0 2024-05-08T21:59:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.243+0000 7f5d87f5e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801065a0 con 0x7f5d801008e0 2024-05-08T21:59:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.244+0000 7f5d85cfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 0x7f5d8006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d8006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d8006fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53484/0 (socket says 172.21.15.64:53484) 2024-05-08T21:59:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 -- 172.21.15.64:0/947752924 learned_addr learned my addr 172.21.15.64:0/947752924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 -- 172.21.15.64:0/947752924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d801008e0 msgr2=0x7f5d801ad640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:59:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d801008e0 0x7f5d801ad640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 -- 172.21.15.64:0/947752924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 msgr2=0x7f5d8006f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 0x7f5d8006f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.245+0000 7f5d854f9700 1 -- 172.21.15.64:0/947752924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0095f0 con 0x7f5d800fff20 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.246+0000 7f5d854f9700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d8006fe90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000b530 tx=0x7f5d7000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.246+0000 7f5d76ffd700 1 -- 172.21.15.64:0/947752924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70011830 con 0x7f5d800fff20 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.246+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d80106880 con 0x7f5d800fff20 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.246+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80106de0 con 0x7f5d800fff20 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.246+0000 7f5d76ffd700 1 -- 172.21.15.64:0/947752924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d70011e50 con 0x7f5d800fff20 2024-05-08T21:59:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.246+0000 7f5d76ffd700 1 -- 172.21.15.64:0/947752924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000f2e0 con 0x7f5d800fff20 2024-05-08T21:59:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.248+0000 7f5d76ffd700 1 -- 172.21.15.64:0/947752924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5d7000f440 con 0x7f5d800fff20 2024-05-08T21:59:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.249+0000 7f5d76ffd700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5d6c0a1120 0x7f5d6c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.249+0000 7f5d76ffd700 1 -- 172.21.15.64:0/947752924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d700cd3a0 con 0x7f5d800fff20 2024-05-08T21:59:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.250+0000 7f5d94118b80 1 Processor -- start 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.250+0000 7f5d94118b80 1 -- start start 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b4eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b4fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b50750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d94118b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00aa60f0 con 0x561f00b4ea80 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d94118b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00aa3ab0 con 0x561f00b4f3e0 2024-05-08T21:59:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d94118b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00a9f3a0 con 0x561f00b502d0 2024-05-08T21:59:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b4fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b4fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:50400/0 (socket says 172.21.15.64:50400) 2024-05-08T21:59:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 -- 172.21.15.64:0/1861716064 learned_addr learned my addr 172.21.15.64:0/1861716064 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 -- 172.21.15.64:0/1861716064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 msgr2=0x561f00b50750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b50750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 -- 172.21.15.64:0/1861716064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 msgr2=0x561f00b4eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b4eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.251+0000 7f5d854f9700 1 -- 172.21.15.64:0/1861716064 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f00920350 con 0x561f00b4f3e0 2024-05-08T21:59:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.252+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5d6c0a1120 0x7f5d6c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.252+0000 7f5d854f9700 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b4fcf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5d70097270 tx=0x7f5d70004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea32be3faa10f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.252+0000 7f5d6bfff700 1 -- 172.21.15.64:0/1861716064 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7009e630 con 0x561f00b4f3e0 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.252+0000 7f5d6bfff700 1 -- 172.21.15.64:0/1861716064 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d700a2c80 con 0x561f00b4f3e0 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.252+0000 7f5d6bfff700 1 -- 172.21.15.64:0/1861716064 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70099f50 con 0x561f00b4f3e0 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.253+0000 7f5d94118b80 1 -- 172.21.15.64:0/1861716064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 msgr2=0x561f00b4fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.253+0000 7f5d94118b80 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b4fcf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5d70097270 tx=0x7f5d70004860 comp rx=0 tx=0).stop 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.253+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5d6c0a1120 0x7f5d6c0a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b5c0 tx=0x7f5d7c003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.254+0000 7f5d94118b80 1 -- 172.21.15.64:0/1861716064 shutdown_connections 2024-05-08T21:59:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.254+0000 7f5d94118b80 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b50750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.254+0000 7f5d94118b80 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b4eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.254+0000 7f5d94118b80 1 --2- 172.21.15.64:0/1861716064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b4fcf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.254+0000 7f5d94118b80 1 -- 172.21.15.64:0/1861716064 >> 172.21.15.64:0/1861716064 conn(0x561f00aad1b0 msgr2=0x561f00b4b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.255+0000 7f5d94118b80 1 -- 172.21.15.64:0/1861716064 shutdown_connections 2024-05-08T21:59:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.260+0000 7f5d94118b80 1 -- 172.21.15.64:0/1861716064 wait complete. 2024-05-08T21:59:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.260+0000 7f5d94118b80 1 -- start start 2024-05-08T21:59:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.260+0000 7f5d94118b80 1 Processor -- start 2024-05-08T21:59:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4ea80 0x561f00b588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4f3e0 0x561f00b58e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00a3b500 con 0x561f00b4f3e0 2024-05-08T21:59:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00a3b1a0 con 0x561f00b4ea80 2024-05-08T21:59:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00922640 con 0x561f00b502d0 2024-05-08T21:59:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d94118b80 10 client.0 _ll_register_callbacks cb 0x561f00a44ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d864fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.261+0000 7f5d85cfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4ea80 0x561f00b588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b5fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48878/0 (socket says 172.21.15.64:48878) 2024-05-08T21:59:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 -- 172.21.15.64:0/134712844 learned_addr learned my addr 172.21.15.64:0/134712844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 -- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4ea80 msgr2=0x561f00b588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4ea80 0x561f00b588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 -- 172.21.15.64:0/134712844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4f3e0 msgr2=0x561f00b58e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:59:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4f3e0 0x561f00b58e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 -- 172.21.15.64:0/134712844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80106de0 con 0x561f00b502d0 2024-05-08T21:59:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d864fb700 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b5fe70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007b20 tx=0x7f5d78007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7287c15913302c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d697fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:59:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.262+0000 7f5d697fa700 1 -- 172.21.15.64:0/134712844 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78017070 con 0x561f00b502d0 2024-05-08T21:59:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.263+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f009f8210 con 0x561f00b502d0 2024-05-08T21:59:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.263+0000 7f5d697fa700 1 -- 172.21.15.64:0/134712844 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d78009e00 con 0x561f00b502d0 2024-05-08T21:59:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.263+0000 7f5d697fa700 1 -- 172.21.15.64:0/134712844 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d78003a00 con 0x561f00b502d0 2024-05-08T21:59:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.263+0000 7f5d94118b80 10 client.25258 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.263+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561f008ccb00 con 0x561f00b502d0 2024-05-08T21:59:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.263+0000 7f5d697fa700 1 -- 172.21.15.64:0/134712844 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f009f8210 con 0x561f00b502d0 2024-05-08T21:59:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d697fa700 1 -- 172.21.15.64:0/134712844 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d7801a030 con 0x561f00b502d0 2024-05-08T21:59:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d94118b80 10 client.25258 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d94118b80 10 client.25258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d94118b80 20 client.25258 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d94118b80 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00b64830 0x561f00b66c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d94118b80 4 client.25258 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f00b1e7b0 con 0x561f00b64830 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.264+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00b64830 0x561f00b66c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.265+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00b64830 0x561f00b66c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813c0b80954064c0 server_cookie=4cea59b1a7a5174d in_seq=0 out_seq=0 2024-05-08T21:59:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.265+0000 7f5d697fa700 10 client.25258 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.265+0000 7f5d697fa700 1 -- 172.21.15.64:0/134712844 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f00b1e7b0 con 0x561f00b64830 2024-05-08T21:59:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.265+0000 7f5d697fa700 10 client.25258 handle_command_reply: tid=0 2024-05-08T21:59:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.265+0000 7f5d94118b80 1 client.25258 shutdown 2024-05-08T21:59:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 msgr2=0x561f00b5fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b5fe70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007b20 tx=0x7f5d78007e30 comp rx=0 tx=0).stop 2024-05-08T21:59:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 shutdown_connections 2024-05-08T21:59:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4ea80 0x561f00b588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00b64830 0x561f00b66c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b502d0 0x561f00b5fe70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 --2- 172.21.15.64:0/134712844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4f3e0 0x561f00b58e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 >> 172.21.15.64:0/134712844 conn(0x561f00aad1b0 msgr2=0x561f00b49860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d697fa700 0 client.25258 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 shutdown_connections 2024-05-08T21:59:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.266+0000 7f5d94118b80 1 -- 172.21.15.64:0/134712844 wait complete. 2024-05-08T21:59:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.267+0000 7f5d94118b80 20 client.25258 trim_cache size 0 max 16384 2024-05-08T21:59:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 Processor -- start 2024-05-08T21:59:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 -- start start 2024-05-08T21:59:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4ea80 0x561f00b66c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b502d0 0x561f00b5fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00b60ff0 con 0x561f00b502d0 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00b58460 con 0x561f00b4f3e0 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.306+0000 7f5d94118b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00b4e6e0 con 0x561f00b4ea80 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.307+0000 7f5d854f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.307+0000 7f5d854f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:50420/0 (socket says 172.21.15.64:50420) 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.307+0000 7f5d854f9700 1 -- 172.21.15.64:0/2596413561 learned_addr learned my addr 172.21.15.64:0/2596413561 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.307+0000 7f5d864fb700 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b502d0 0x561f00b5fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d864fb700 1 -- 172.21.15.64:0/2596413561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4ea80 msgr2=0x561f00b66c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d864fb700 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4ea80 0x561f00b66c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d864fb700 1 -- 172.21.15.64:0/2596413561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 msgr2=0x561f00b506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d864fb700 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d864fb700 1 -- 172.21.15.64:0/2596413561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f00a65500 con 0x561f00b502d0 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d864fb700 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b502d0 0x561f00b5fd90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5d78013410 tx=0x7f5d7800d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa17b5feccfe92ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d74ff9700 1 -- 172.21.15.64:0/2596413561 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78017070 con 0x561f00b502d0 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d74ff9700 1 -- 172.21.15.64:0/2596413561 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7800f040 con 0x561f00b502d0 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.308+0000 7f5d74ff9700 1 -- 172.21.15.64:0/2596413561 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78004c60 con 0x561f00b502d0 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.309+0000 7f5d94118b80 1 -- 172.21.15.64:0/2596413561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b502d0 msgr2=0x561f00b5fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.309+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b502d0 0x561f00b5fd90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5d78013410 tx=0x7f5d7800d470 comp rx=0 tx=0).stop 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 -- 172.21.15.64:0/2596413561 shutdown_connections 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b502d0 0x561f00b5fd90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4ea80 0x561f00b66c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2596413561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b4f3e0 0x561f00b506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 -- 172.21.15.64:0/2596413561 >> 172.21.15.64:0/2596413561 conn(0x561f00aad1b0 msgr2=0x561f00b568a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 -- 172.21.15.64:0/2596413561 shutdown_connections 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 -- 172.21.15.64:0/2596413561 wait complete. 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 -- start start 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.310+0000 7f5d94118b80 1 Processor -- start 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b5d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4f3e0 0x561f00b5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b502d0 0x561f00b5e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00b5ec80 con 0x561f00b4ea80 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00b5edf0 con 0x561f00b502d0 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f00b5efb0 con 0x561f00b4f3e0 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d94118b80 10 client.0 _ll_register_callbacks cb 0x561f00a44ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d85cfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b5d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d85cfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b5d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53512/0 (socket says 172.21.15.64:53512) 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.311+0000 7f5d85cfa700 1 -- 172.21.15.64:0/2194129791 learned_addr learned my addr 172.21.15.64:0/2194129791 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d85cfa700 1 -- 172.21.15.64:0/2194129791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4f3e0 msgr2=0x561f00b5d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4f3e0 0x561f00b5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d85cfa700 1 -- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b502d0 msgr2=0x561f00b5e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b502d0 0x561f00b5e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d85cfa700 1 -- 172.21.15.64:0/2194129791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f008ccb00 con 0x561f00b4ea80 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d85cfa700 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b5d1f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0098e0 tx=0x7f5d7c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2caa83222a99d29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d697fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d697fa700 1 -- 172.21.15.64:0/2194129791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c02a040 con 0x561f00b4ea80 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.312+0000 7f5d697fa700 1 -- 172.21.15.64:0/2194129791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d7c005390 con 0x561f00b4ea80 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.313+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d700a2c80 con 0x561f00b4ea80 2024-05-08T21:59:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.313+0000 7f5d697fa700 1 -- 172.21.15.64:0/2194129791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7c005550 con 0x561f00b4ea80 2024-05-08T21:59:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.313+0000 7f5d94118b80 10 client.15615 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.313+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561f00924c70 con 0x561f00b4ea80 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.313+0000 7f5d697fa700 1 -- 172.21.15.64:0/2194129791 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d700a2c80 con 0x561f00b4ea80 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.313+0000 7f5d697fa700 1 -- 172.21.15.64:0/2194129791 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d7c037070 con 0x561f00b4ea80 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d94118b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d94118b80 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d94118b80 20 client.15615 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00bff3a0 0x561f00c017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d94118b80 4 client.15615 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f00a97e60 con 0x561f00bff3a0 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d854f9700 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00bff3a0 0x561f00c017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d697fa700 10 client.15615 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.314+0000 7f5d854f9700 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00bff3a0 0x561f00c017c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06f7dd8ada37ff6 server_cookie=5776bd478b2fad7b in_seq=0 out_seq=0 2024-05-08T21:59:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.340+0000 7f5d697fa700 1 -- 172.21.15.64:0/2194129791 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f00a97e60 con 0x561f00bff3a0 2024-05-08T21:59:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.340+0000 7f5d697fa700 10 client.15615 handle_command_reply: tid=0 2024-05-08T21:59:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.340+0000 7f5d94118b80 1 client.15615 shutdown 2024-05-08T21:59:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.340+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 msgr2=0x561f00b5d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b5d1f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0098e0 tx=0x7f5d7c009310 comp rx=0 tx=0).stop 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 shutdown_connections 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f00b502d0 0x561f00b5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561f00bff3a0 0x561f00c017c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f00b4ea80 0x561f00b5d1f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 --2- 172.21.15.64:0/2194129791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f00b4f3e0 0x561f00b5d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d697fa700 0 client.15615 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 >> 172.21.15.64:0/2194129791 conn(0x561f00aad1b0 msgr2=0x561f00b6b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 shutdown_connections 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 1 -- 172.21.15.64:0/2194129791 wait complete. 2024-05-08T21:59:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.341+0000 7f5d94118b80 20 client.15615 trim_cache size 0 max 16384 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5d6c0a1120 msgr2=0x7f5d6c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5d6c0a1120 0x7f5d6c0a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b5c0 tx=0x7f5d7c003790 comp rx=0 tx=0).stop 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 msgr2=0x7f5d8006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d8006fe90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000b530 tx=0x7f5d7000b8f0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 shutdown_connections 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5d6c0a1120 0x7f5d6c0a35e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d800ff1c0 0x7f5d8006f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d801008e0 0x7f5d801ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 --2- 172.21.15.64:0/947752924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d800fff20 0x7f5d8006fe90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.343+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 >> 172.21.15.64:0/947752924 conn(0x7f5d800faaf0 msgr2=0x7f5d80101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.344+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 shutdown_connections 2024-05-08T21:59:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:03.344+0000 7f5d87f5e700 1 -- 172.21.15.64:0/947752924 wait complete. 2024-05-08T21:59:04.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:04 smithi064 ceph-mon[42500]: pgmap v392: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 854 op/s 2024-05-08T21:59:04.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:04 smithi160 ceph-mon[46962]: pgmap v392: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 854 op/s 2024-05-08T21:59:04.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:04 smithi186 ceph-mon[46619]: pgmap v392: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 854 op/s 2024-05-08T21:59:05.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:05 smithi064 ceph-mon[42500]: pgmap v393: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 956 op/s 2024-05-08T21:59:05.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:05 smithi160 ceph-mon[46962]: pgmap v393: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 956 op/s 2024-05-08T21:59:05.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:05 smithi186 ceph-mon[46619]: pgmap v393: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 956 op/s 2024-05-08T21:59:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 255.0 s, 2232.8 tps, lat 44.902 ms stddev 51.814 2024-05-08T21:59:07.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:07 smithi186 ceph-mon[46619]: pgmap v394: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 860 op/s 2024-05-08T21:59:07.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:07 smithi064 ceph-mon[42500]: pgmap v394: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 860 op/s 2024-05-08T21:59:07.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:07 smithi160 ceph-mon[46962]: pgmap v394: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 860 op/s 2024-05-08T21:59:09.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:09 smithi186 ceph-mon[46619]: pgmap v395: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 886 op/s 2024-05-08T21:59:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:09 smithi064 ceph-mon[42500]: pgmap v395: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 886 op/s 2024-05-08T21:59:09.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:09 smithi160 ceph-mon[46962]: pgmap v395: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 886 op/s 2024-05-08T21:59:11.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 260.0 s, 2745.4 tps, lat 36.420 ms stddev 43.540 2024-05-08T21:59:11.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:11 smithi186 ceph-mon[46619]: pgmap v396: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 964 op/s 2024-05-08T21:59:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:11 smithi064 ceph-mon[42500]: pgmap v396: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 964 op/s 2024-05-08T21:59:11.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:11 smithi160 ceph-mon[46962]: pgmap v396: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 964 op/s 2024-05-08T21:59:13.642 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:13 smithi186 ceph-mon[46619]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 941 op/s 2024-05-08T21:59:13.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:13 smithi064 ceph-mon[42500]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 941 op/s 2024-05-08T21:59:13.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:13 smithi160 ceph-mon[46962]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 941 op/s 2024-05-08T21:59:15.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:15 smithi186 ceph-mon[46619]: pgmap v398: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 MiB/s wr, 1.35k op/s 2024-05-08T21:59:15.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:15 smithi064 ceph-mon[42500]: pgmap v398: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 MiB/s wr, 1.35k op/s 2024-05-08T21:59:15.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:15 smithi160 ceph-mon[46962]: pgmap v398: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 MiB/s wr, 1.35k op/s 2024-05-08T21:59:16.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 265.0 s, 1876.4 tps, lat 52.881 ms stddev 89.108 2024-05-08T21:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.525+0000 7f309d538700 1 -- 172.21.15.64:0/2323558975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3098100890 msgr2=0x7f3098105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.525+0000 7f309d538700 1 --2- 172.21.15.64:0/2323558975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3098100890 0x7f3098105130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f309000b0a0 tx=0x7f309000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 -- 172.21.15.64:0/2323558975 shutdown_connections 2024-05-08T21:59:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 --2- 172.21.15.64:0/2323558975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30980feda0 0x7f30980ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 --2- 172.21.15.64:0/2323558975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3098100890 0x7f3098105130 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 --2- 172.21.15.64:0/2323558975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 0x7f3098100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 -- 172.21.15.64:0/2323558975 >> 172.21.15.64:0/2323558975 conn(0x7f30980fa6a0 msgr2=0x7f30980fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 -- 172.21.15.64:0/2323558975 shutdown_connections 2024-05-08T21:59:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.531+0000 7f309d538700 1 -- 172.21.15.64:0/2323558975 wait complete. 2024-05-08T21:59:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.532+0000 7f309d538700 1 Processor -- start 2024-05-08T21:59:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.532+0000 7f309d538700 1 -- start start 2024-05-08T21:59:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f309d538700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 0x7f309819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f309d538700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 0x7f309819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f309d538700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3098100890 0x7f30981a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f309d538700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30981a5180 con 0x7f30980feda0 2024-05-08T21:59:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f309d538700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30981a52c0 con 0x7f3098100890 2024-05-08T21:59:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f309d538700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30981a5450 con 0x7f30980ffed0 2024-05-08T21:59:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 0x7f309819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 0x7f309819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34156/0 (socket says 172.21.15.64:34156) 2024-05-08T21:59:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.533+0000 7f3096ffd700 1 -- 172.21.15.64:0/2932521304 learned_addr learned my addr 172.21.15.64:0/2932521304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.534+0000 7f30977fe700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3098100890 0x7f30981a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.534+0000 7f30967fc700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 0x7f309819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.535+0000 7f30967fc700 1 -- 172.21.15.64:0/2932521304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3098100890 msgr2=0x7f30981a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.535+0000 7f30967fc700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3098100890 0x7f30981a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.535+0000 7f30967fc700 1 -- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 msgr2=0x7f309819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.535+0000 7f30967fc700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 0x7f309819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.535+0000 7f30967fc700 1 -- 172.21.15.64:0/2932521304 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3090009d00 con 0x7f30980ffed0 2024-05-08T21:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.535+0000 7f3096ffd700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 0x7f309819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.536+0000 7f30967fc700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 0x7f309819dbe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b450 tx=0x7f308c00b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.536+0000 7f307ffff700 1 -- 172.21.15.64:0/2932521304 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011830 con 0x7f30980ffed0 2024-05-08T21:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.536+0000 7f307ffff700 1 -- 172.21.15.64:0/2932521304 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308c011e50 con 0x7f30980ffed0 2024-05-08T21:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.536+0000 7f307ffff700 1 -- 172.21.15.64:0/2932521304 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c010e00 con 0x7f30980ffed0 2024-05-08T21:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.536+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30981a5730 con 0x7f30980ffed0 2024-05-08T21:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.536+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30981a5c70 con 0x7f30980ffed0 2024-05-08T21:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.540+0000 7f307ffff700 1 -- 172.21.15.64:0/2932521304 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f308c010380 con 0x7f30980ffed0 2024-05-08T21:59:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 Processor -- start 2024-05-08T21:59:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f307ffff700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30800a11a0 0x7f30800a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 -- start start 2024-05-08T21:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f307ffff700 1 -- 172.21.15.64:0/2932521304 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f308c0cb6b0 con 0x7f30980ffed0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 0x560cf45c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c73e0 0x560cf45c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf451e0f0 con 0x560cf45c82d0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30977fe700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30800a11a0 0x7f30800a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.542+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf451bab0 con 0x560cf45c6a80 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 0x560cf45c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 0x560cf45c6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35538/0 (socket says 172.21.15.64:35538) 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 -- 172.21.15.64:0/2935603620 learned_addr learned my addr 172.21.15.64:0/2935603620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f30967fc700 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c73e0 0x560cf45c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f30a96f2b80 1 -- 172.21.15.64:0/2935603620 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45173a0 con 0x560cf45c73e0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f30977fe700 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 -- 172.21.15.64:0/2935603620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c73e0 msgr2=0x560cf45c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c73e0 0x560cf45c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 -- 172.21.15.64:0/2935603620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 msgr2=0x560cf45c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f3096ffd700 1 -- 172.21.15.64:0/2935603620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cf4398350 con 0x560cf45c6a80 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.543+0000 7f30977fe700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30800a11a0 0x7f30800a3660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f30900062a0 tx=0x7f30900077e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.544+0000 7f3096ffd700 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 0x560cf45c6ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3088009fb0 tx=0x7f308800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0bb3b6585a1dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.544+0000 7f307d7fa700 1 -- 172.21.15.64:0/2935603620 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308800ed40 con 0x560cf45c6a80 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.544+0000 7f307d7fa700 1 -- 172.21.15.64:0/2935603620 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308800eea0 con 0x560cf45c6a80 2024-05-08T21:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.544+0000 7f307d7fa700 1 -- 172.21.15.64:0/2935603620 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880127f0 con 0x560cf45c6a80 2024-05-08T21:59:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.545+0000 7f30a96f2b80 1 -- 172.21.15.64:0/2935603620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 msgr2=0x560cf45c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.545+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 0x560cf45c6ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3088009fb0 tx=0x7f308800e4b0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 -- 172.21.15.64:0/2935603620 shutdown_connections 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c6a80 0x560cf45c6ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/2935603620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c73e0 0x560cf45c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 -- 172.21.15.64:0/2935603620 >> 172.21.15.64:0/2935603620 conn(0x560cf45251b0 msgr2=0x560cf45c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 -- 172.21.15.64:0/2935603620 shutdown_connections 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.547+0000 7f30a96f2b80 1 -- 172.21.15.64:0/2935603620 wait complete. 2024-05-08T21:59:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 -- start start 2024-05-08T21:59:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 Processor -- start 2024-05-08T21:59:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf44b3500 con 0x560cf45c82d0 2024-05-08T21:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf44b31a0 con 0x560cf45c73e0 2024-05-08T21:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf439a640 con 0x560cf45c6a80 2024-05-08T21:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30a96f2b80 10 client.0 _ll_register_callbacks cb 0x560cf44bcad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35552/0 (socket says 172.21.15.64:35552) 2024-05-08T21:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.548+0000 7f30967fc700 1 -- 172.21.15.64:0/1799094448 learned_addr learned my addr 172.21.15.64:0/1799094448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f30967fc700 1 -- 172.21.15.64:0/1799094448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 msgr2=0x560cf45e18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f30967fc700 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45e18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f30967fc700 1 -- 172.21.15.64:0/1799094448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 msgr2=0x560cf45e8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f30967fc700 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f30967fc700 1 -- 172.21.15.64:0/1799094448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3088009be0 con 0x560cf45c73e0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f30967fc700 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e1e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f308c0c9210 tx=0x7f308c0093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2000b9dc1f51f1b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f307aff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f307aff5700 1 -- 172.21.15.64:0/1799094448 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c098200 con 0x560cf45c73e0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.549+0000 7f307aff5700 1 -- 172.21.15.64:0/1799094448 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f308c09c940 con 0x560cf45c73e0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.550+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cf4470210 con 0x560cf45c73e0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.550+0000 7f307aff5700 1 -- 172.21.15.64:0/1799094448 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f308c0a0f90 con 0x560cf45c73e0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.550+0000 7f30a96f2b80 10 client.25352 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.550+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x560cf4344b00 con 0x560cf45c73e0 2024-05-08T21:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f307aff5700 1 -- 172.21.15.64:0/1799094448 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cf4470210 con 0x560cf45c73e0 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f307aff5700 1 -- 172.21.15.64:0/1799094448 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f308c099b20 con 0x560cf45c73e0 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f30a96f2b80 10 client.25352 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f30a96f2b80 10 client.25352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f30a96f2b80 20 client.25352 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf45ed750 0x560cf45efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f30a96f2b80 4 client.25352 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cf45967b0 con 0x560cf45ed750 2024-05-08T21:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f3096ffd700 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf45ed750 0x560cf45efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f307aff5700 10 client.25352 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.551+0000 7f3096ffd700 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf45ed750 0x560cf45efb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a56ad7cf7e28de server_cookie=8c74e51eeb0b7394 in_seq=0 out_seq=0 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.552+0000 7f307aff5700 1 -- 172.21.15.64:0/1799094448 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cf45967b0 con 0x560cf45ed750 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.552+0000 7f307aff5700 10 client.25352 handle_command_reply: tid=0 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.552+0000 7f30a96f2b80 1 client.25352 shutdown 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.552+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 msgr2=0x560cf45e1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.552+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e1e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f308c0c9210 tx=0x7f308c0093b0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 shutdown_connections 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf45ed750 0x560cf45efb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c82d0 0x560cf45e8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45e18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f307aff5700 0 client.25352 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1799094448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e1e00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 >> 172.21.15.64:0/1799094448 conn(0x560cf45251b0 msgr2=0x560cf45c1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 shutdown_connections 2024-05-08T21:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1799094448 wait complete. 2024-05-08T21:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.553+0000 7f30a96f2b80 20 client.25352 trim_cache size 0 max 16384 2024-05-08T21:59:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.579+0000 7f30a96f2b80 1 Processor -- start 2024-05-08T21:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.579+0000 7f30a96f2b80 1 -- start start 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.579+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45efb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.579+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c73e0 0x560cf45e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c82d0 0x560cf45c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45e27c0 con 0x560cf45c73e0 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45f4de0 con 0x560cf45c82d0 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45e2ab0 con 0x560cf45c6a80 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c82d0 0x560cf45c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45efb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:37936/0 (socket says 172.21.15.64:37936) 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c82d0 0x560cf45c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35564/0 (socket says 172.21.15.64:35564) 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 -- 172.21.15.64:0/1039988184 learned_addr learned my addr 172.21.15.64:0/1039988184 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f30967fc700 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c73e0 0x560cf45e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 -- 172.21.15.64:0/1039988184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c82d0 msgr2=0x560cf45c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c82d0 0x560cf45c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 -- 172.21.15.64:0/1039988184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c73e0 msgr2=0x560cf45e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c73e0 0x560cf45e8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 -- 172.21.15.64:0/1039988184 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cf44dd500 con 0x560cf45c6a80 2024-05-08T21:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f3096ffd700 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45efb70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3088009fb0 tx=0x7f3088008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6420ee864b9533af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.580+0000 7f307dffb700 1 -- 172.21.15.64:0/1039988184 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088023b70 con 0x560cf45c6a80 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.581+0000 7f307dffb700 1 -- 172.21.15.64:0/1039988184 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3088023cd0 con 0x560cf45c6a80 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.581+0000 7f307dffb700 1 -- 172.21.15.64:0/1039988184 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088013930 con 0x560cf45c6a80 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1039988184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 msgr2=0x560cf45efb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45efb70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3088009fb0 tx=0x7f3088008040 comp rx=0 tx=0).stop 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1039988184 shutdown_connections 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c82d0 0x560cf45c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c6a80 0x560cf45efb70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/1039988184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c73e0 0x560cf45e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1039988184 >> 172.21.15.64:0/1039988184 conn(0x560cf45251b0 msgr2=0x560cf45e0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1039988184 shutdown_connections 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.582+0000 7f30a96f2b80 1 -- 172.21.15.64:0/1039988184 wait complete. 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 -- start start 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 Processor -- start 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c6a80 0x560cf45e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c82d0 0x560cf45e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45e7fb0 con 0x560cf45c6a80 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45e8120 con 0x560cf45c73e0 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf45e82e0 con 0x560cf45c82d0 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30a96f2b80 10 client.0 _ll_register_callbacks cb 0x560cf44bcad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35574/0 (socket says 172.21.15.64:35574) 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.583+0000 7f30967fc700 1 -- 172.21.15.64:0/583842808 learned_addr learned my addr 172.21.15.64:0/583842808 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30967fc700 1 -- 172.21.15.64:0/583842808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c82d0 msgr2=0x560cf45e7a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30967fc700 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c82d0 0x560cf45e7a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30967fc700 1 -- 172.21.15.64:0/583842808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c6a80 msgr2=0x560cf45e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30967fc700 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c6a80 0x560cf45e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30967fc700 1 -- 172.21.15.64:0/583842808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cf4398350 con 0x560cf45c73e0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30967fc700 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e4af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f308c0c9530 tx=0x7f308c0b1500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af580dec3e968b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f307aff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f307aff5700 1 -- 172.21.15.64:0/583842808 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0a0d10 con 0x560cf45c73e0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f307aff5700 1 -- 172.21.15.64:0/583842808 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f308c098000 con 0x560cf45c73e0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.584+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f308800eea0 con 0x560cf45c73e0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f307aff5700 1 -- 172.21.15.64:0/583842808 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f308c0981c0 con 0x560cf45c73e0 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 10 client.25358 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x560cf439cc70 con 0x560cf45c73e0 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f307aff5700 1 -- 172.21.15.64:0/583842808 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308800eea0 con 0x560cf45c73e0 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f307aff5700 1 -- 172.21.15.64:0/583842808 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f308c097a70 con 0x560cf45c73e0 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 10 client.25358 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 10 client.25358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 20 client.25358 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf4655260 0x560cf4657680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 4 client.25358 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560cf450fe60 con 0x560cf4655260 2024-05-08T21:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.585+0000 7f3096ffd700 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf4655260 0x560cf4657680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.586+0000 7f307aff5700 10 client.25358 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.586+0000 7f3096ffd700 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf4655260 0x560cf4657680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0795e5bfe610c64 server_cookie=73972cd9c5a843d4 in_seq=0 out_seq=0 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f307aff5700 1 -- 172.21.15.64:0/583842808 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560cf450fe60 con 0x560cf4655260 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f307aff5700 10 client.25358 handle_command_reply: tid=0 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 client.25358 shutdown 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 msgr2=0x560cf45e4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e4af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f308c0c9530 tx=0x7f308c0b1500 comp rx=0 tx=0).stop 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 shutdown_connections 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560cf4655260 0x560cf4657680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cf45c82d0 0x560cf45e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f307aff5700 0 client.25358 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cf45c6a80 0x560cf45e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 --2- 172.21.15.64:0/583842808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560cf45c73e0 0x560cf45e4af0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.601+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 >> 172.21.15.64:0/583842808 conn(0x560cf45251b0 msgr2=0x560cf45e0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 shutdown_connections 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f30a96f2b80 1 -- 172.21.15.64:0/583842808 wait complete. 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f30a96f2b80 20 client.25358 trim_cache size 0 max 16384 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:59:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:59:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30800a11a0 msgr2=0x7f30800a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30800a11a0 0x7f30800a3660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f30900062a0 tx=0x7f30900077e0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 msgr2=0x7f309819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 0x7f309819dbe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b450 tx=0x7f308c00b760 comp rx=0 tx=0).stop 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 shutdown_connections 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f30800a11a0 0x7f30800a3660 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3098100890 0x7f30981a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30980feda0 0x7f309819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 --2- 172.21.15.64:0/2932521304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30980ffed0 0x7f309819dbe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 >> 172.21.15.64:0/2932521304 conn(0x7f30980fa6a0 msgr2=0x7f3098101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 shutdown_connections 2024-05-08T21:59:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:16.602+0000 7f309d538700 1 -- 172.21.15.64:0/2932521304 wait complete. 2024-05-08T21:59:17.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:17 smithi186 ceph-mon[46619]: pgmap v399: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 1.47k op/s 2024-05-08T21:59:17.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:17 smithi064 ceph-mon[42500]: pgmap v399: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 1.47k op/s 2024-05-08T21:59:17.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:17 smithi160 ceph-mon[46962]: pgmap v399: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 1.47k op/s 2024-05-08T21:59:19.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:19 smithi186 ceph-mon[46619]: pgmap v400: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 44 MiB/s wr, 1.90k op/s 2024-05-08T21:59:19.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:19 smithi064 ceph-mon[42500]: pgmap v400: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 44 MiB/s wr, 1.90k op/s 2024-05-08T21:59:19.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:19 smithi160 ceph-mon[46962]: pgmap v400: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 44 MiB/s wr, 1.90k op/s 2024-05-08T21:59:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 270.0 s, 2367.1 tps, lat 42.626 ms stddev 49.960 2024-05-08T21:59:21.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:21 smithi186 ceph-mon[46619]: pgmap v401: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 45 MiB/s wr, 1.98k op/s 2024-05-08T21:59:21.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:21 smithi064 ceph-mon[42500]: pgmap v401: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 45 MiB/s wr, 1.98k op/s 2024-05-08T21:59:21.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:21 smithi160 ceph-mon[46962]: pgmap v401: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 45 MiB/s wr, 1.98k op/s 2024-05-08T21:59:23.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:23 smithi064 ceph-mon[42500]: pgmap v402: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 1.97k op/s 2024-05-08T21:59:23.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:23 smithi160 ceph-mon[46962]: pgmap v402: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 1.97k op/s 2024-05-08T21:59:23.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:23 smithi186 ceph-mon[46619]: pgmap v402: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 1.97k op/s 2024-05-08T21:59:25.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:25 smithi064 ceph-mon[42500]: pgmap v403: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.10k op/s 2024-05-08T21:59:25.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:25 smithi160 ceph-mon[46962]: pgmap v403: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.10k op/s 2024-05-08T21:59:25.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:25 smithi186 ceph-mon[46619]: pgmap v403: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.10k op/s 2024-05-08T21:59:26.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 275.0 s, 3020.4 tps, lat 33.127 ms stddev 37.011 2024-05-08T21:59:27.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:27 smithi064 ceph-mon[42500]: pgmap v404: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 1.62k op/s 2024-05-08T21:59:27.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:27 smithi160 ceph-mon[46962]: pgmap v404: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 1.62k op/s 2024-05-08T21:59:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:27 smithi186 ceph-mon[46619]: pgmap v404: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 1.62k op/s 2024-05-08T21:59:29.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:29 smithi064 ceph-mon[42500]: pgmap v405: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 1.35k op/s 2024-05-08T21:59:29.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 -- 172.21.15.64:0/30580337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb60102190 msgr2=0x7fcb601025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 --2- 172.21.15.64:0/30580337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb60102190 0x7fcb601025f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009990 tx=0x7fcb50009ca0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 -- 172.21.15.64:0/30580337 shutdown_connections 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 --2- 172.21.15.64:0/30580337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb600ff1c0 0x7fcb600ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 --2- 172.21.15.64:0/30580337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb60102b30 0x7fcb601093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 --2- 172.21.15.64:0/30580337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb60102190 0x7fcb601025f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 -- 172.21.15.64:0/30580337 >> 172.21.15.64:0/30580337 conn(0x7fcb600faaf0 msgr2=0x7fcb600fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 -- 172.21.15.64:0/30580337 shutdown_connections 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.760+0000 7fcb66b84700 1 -- 172.21.15.64:0/30580337 wait complete. 2024-05-08T21:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.761+0000 7fcb66b84700 1 Processor -- start 2024-05-08T21:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 -- start start 2024-05-08T21:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600ff1c0 0x7fcb60101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb60102190 0x7fcb60100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 0x7fcb60100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60100e20 con 0x7fcb600ff1c0 2024-05-08T21:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60100f90 con 0x7fcb60102b30 2024-05-08T21:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb66b84700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601a0ed0 con 0x7fcb60102190 2024-05-08T21:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb65121700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 0x7fcb60100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb65121700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 0x7fcb60100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45254/0 (socket says 172.21.15.64:45254) 2024-05-08T21:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.762+0000 7fcb65121700 1 -- 172.21.15.64:0/1317891583 learned_addr learned my addr 172.21.15.64:0/1317891583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb65121700 1 -- 172.21.15.64:0/1317891583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb60102190 msgr2=0x7fcb60100140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb65121700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb60102190 0x7fcb60100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb65121700 1 -- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600ff1c0 msgr2=0x7fcb60101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb65121700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600ff1c0 0x7fcb60101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb65121700 1 -- 172.21.15.64:0/1317891583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500095f0 con 0x7fcb60102b30 2024-05-08T21:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb65121700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 0x7fcb60100680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800ef30 tx=0x7fcb5800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb5dffb700 1 -- 172.21.15.64:0/1317891583 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58018070 con 0x7fcb60102b30 2024-05-08T21:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb5dffb700 1 -- 172.21.15.64:0/1317891583 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5800f040 con 0x7fcb60102b30 2024-05-08T21:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.765+0000 7fcb5dffb700 1 -- 172.21.15.64:0/1317891583 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800cde0 con 0x7fcb60102b30 2024-05-08T21:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.766+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb601a1100 con 0x7fcb60102b30 2024-05-08T21:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.766+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb601a15c0 con 0x7fcb60102b30 2024-05-08T21:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.770+0000 7fcb5dffb700 1 -- 172.21.15.64:0/1317891583 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcb58004ad0 con 0x7fcb60102b30 2024-05-08T21:59:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.770+0000 7fcb72d3eb80 1 Processor -- start 2024-05-08T21:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.773+0000 7fcb72d3eb80 1 -- start start 2024-05-08T21:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.773+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0040f0 con 0x55fefb0b5290 2024-05-08T21:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb001ab0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefaffd3a0 con 0x55fefb0b6ae0 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5dffb700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcb4c0a11a0 0x7fcb4c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44920/0 (socket says 172.21.15.64:44920) 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 -- 172.21.15.64:0/731391285 learned_addr learned my addr 172.21.15.64:0/731391285 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb64920700 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5dffb700 1 -- 172.21.15.64:0/1317891583 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcb580cbcf0 con 0x7fcb60102b30 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 -- 172.21.15.64:0/731391285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 msgr2=0x55fefb0b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 -- 172.21.15.64:0/731391285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 msgr2=0x55fefb0b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb65121700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcb4c0a11a0 0x7fcb4c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.774+0000 7fcb5ffff700 1 -- 172.21.15.64:0/731391285 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefae7e350 con 0x55fefb0b6ae0 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.775+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0b6f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb50004800 tx=0x7fcb50009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aeee173e66cdafe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.775+0000 7fcb56ffd700 1 -- 172.21.15.64:0/731391285 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000bd50 con 0x55fefb0b6ae0 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.775+0000 7fcb56ffd700 1 -- 172.21.15.64:0/731391285 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5000beb0 con 0x55fefb0b6ae0 2024-05-08T21:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.775+0000 7fcb56ffd700 1 -- 172.21.15.64:0/731391285 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50017650 con 0x55fefb0b6ae0 2024-05-08T21:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.776+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/731391285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 msgr2=0x55fefb0b6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.776+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0b6f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb50004800 tx=0x7fcb50009310 comp rx=0 tx=0).stop 2024-05-08T21:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.776+0000 7fcb65121700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcb4c0a11a0 0x7fcb4c0a3660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb58094ed0 tx=0x7fcb5800cf40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/731391285 shutdown_connections 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0b6f60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/731391285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/731391285 >> 172.21.15.64:0/731391285 conn(0x55fefb00b1b0 msgr2=0x55fefb0a9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/731391285 shutdown_connections 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/731391285 wait complete. 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.777+0000 7fcb72d3eb80 1 -- start start 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 Processor -- start 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefaf99500 con 0x55fefb0b5290 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefaf991a0 con 0x55fefb0b6ae0 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefae80640 con 0x55fefb0b5bf0 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.778+0000 7fcb72d3eb80 10 client.0 _ll_register_callbacks cb 0x55fefafa2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53250/0 (socket says 172.21.15.64:53250) 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 -- 172.21.15.64:0/89880199 learned_addr learned my addr 172.21.15.64:0/89880199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 -- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 msgr2=0x55fefb0c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 -- 172.21.15.64:0/89880199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 msgr2=0x55fefb0bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.779+0000 7fcb64920700 1 -- 172.21.15.64:0/89880199 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb60109ce0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb64920700 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0bf610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcb48011690 tx=0x7fcb480119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6251f7605d731b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb3ffff700 1 -- 172.21.15.64:0/89880199 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480097e0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb3ffff700 1 -- 172.21.15.64:0/89880199 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcb48004a30 con 0x55fefb0b5bf0 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fefaf56210 con 0x55fefb0b5bf0 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb3ffff700 1 -- 172.21.15.64:0/89880199 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb48012660 con 0x55fefb0b5bf0 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb72d3eb80 10 client.25276 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.780+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fefae2ab00 con 0x55fefb0b5bf0 2024-05-08T21:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.781+0000 7fcb3ffff700 1 -- 172.21.15.64:0/89880199 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fefaf56210 con 0x55fefb0b5bf0 2024-05-08T21:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.781+0000 7fcb3ffff700 1 -- 172.21.15.64:0/89880199 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb48024a80 con 0x55fefb0b5bf0 2024-05-08T21:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb72d3eb80 10 client.25276 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb72d3eb80 10 client.25276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb72d3eb80 20 client.25276 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0c95d0 0x55fefb0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb72d3eb80 4 client.25276 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fefb07c7b0 con 0x55fefb0c95d0 2024-05-08T21:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.782+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0c95d0 0x55fefb0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.783+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0c95d0 0x55fefb0cb9f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66949abfb2266b52 server_cookie=e63fecc0c4a4af75 in_seq=0 out_seq=0 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.783+0000 7fcb3ffff700 10 client.25276 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.784+0000 7fcb3ffff700 1 -- 172.21.15.64:0/89880199 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fefb07c7b0 con 0x55fefb0c95d0 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.784+0000 7fcb3ffff700 10 client.25276 handle_command_reply: tid=0 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.784+0000 7fcb72d3eb80 1 client.25276 shutdown 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.784+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 msgr2=0x55fefb0bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.784+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0bf610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcb48011690 tx=0x7fcb480119a0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 shutdown_connections 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0c95d0 0x55fefb0cb9f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/89880199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0bf610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 >> 172.21.15.64:0/89880199 conn(0x55fefb00b1b0 msgr2=0x55fefb0a7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb3ffff700 0 client.25276 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 shutdown_connections 2024-05-08T21:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/89880199 wait complete. 2024-05-08T21:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.785+0000 7fcb72d3eb80 20 client.25276 trim_cache size 0 max 16384 2024-05-08T21:59:29.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:29 smithi160 ceph-mon[46962]: pgmap v405: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 1.35k op/s 2024-05-08T21:59:29.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.826+0000 7fcb72d3eb80 1 Processor -- start 2024-05-08T21:59:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 -- start start 2024-05-08T21:59:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0d7ac0 con 0x55fefb0b5290 2024-05-08T21:59:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0d0bc0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.832+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0bccf0 con 0x55fefb0b6ae0 2024-05-08T21:59:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.837+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.837+0000 7fcb64920700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.837+0000 7fcb64920700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45268/0 (socket says 172.21.15.64:45268) 2024-05-08T21:59:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.837+0000 7fcb64920700 1 -- 172.21.15.64:0/2605194153 learned_addr learned my addr 172.21.15.64:0/2605194153 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.838+0000 7fcb64920700 1 -- 172.21.15.64:0/2605194153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 msgr2=0x55fefb0d7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.838+0000 7fcb64920700 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0d7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.838+0000 7fcb64920700 1 -- 172.21.15.64:0/2605194153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 msgr2=0x55fefb0cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.838+0000 7fcb64920700 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.838+0000 7fcb64920700 1 -- 172.21.15.64:0/2605194153 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefafc3500 con 0x55fefb0b5bf0 2024-05-08T21:59:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.839+0000 7fcb64920700 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b5670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb48001eb0 tx=0x7fcb48011f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30b7fe403c85eef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.839+0000 7fcb577fe700 1 -- 172.21.15.64:0/2605194153 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48004830 con 0x55fefb0b5bf0 2024-05-08T21:59:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.839+0000 7fcb577fe700 1 -- 172.21.15.64:0/2605194153 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48004e50 con 0x55fefb0b5bf0 2024-05-08T21:59:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.839+0000 7fcb577fe700 1 -- 172.21.15.64:0/2605194153 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480052a0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.840+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/2605194153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 msgr2=0x55fefb0b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.840+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b5670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb48001eb0 tx=0x7fcb48011f40 comp rx=0 tx=0).stop 2024-05-08T21:59:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.847+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/2605194153 shutdown_connections 2024-05-08T21:59:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.847+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b6ae0 0x55fefb0d7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.847+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.847+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/2605194153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b5bf0 0x55fefb0b5670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.847+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/2605194153 >> 172.21.15.64:0/2605194153 conn(0x55fefb00b1b0 msgr2=0x55fefb0bd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.851+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/2605194153 shutdown_connections 2024-05-08T21:59:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.859+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/2605194153 wait complete. 2024-05-08T21:59:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.859+0000 7fcb72d3eb80 1 -- start start 2024-05-08T21:59:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.859+0000 7fcb72d3eb80 1 Processor -- start 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0c52a0 con 0x55fefb0b5290 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0c5410 con 0x55fefb0b6ae0 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefb0c55d0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.860+0000 7fcb72d3eb80 10 client.0 _ll_register_callbacks cb 0x55fefafa2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.861+0000 7fcb64920700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.861+0000 7fcb64920700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53264/0 (socket says 172.21.15.64:53264) 2024-05-08T21:59:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.861+0000 7fcb64920700 1 -- 172.21.15.64:0/1722557768 learned_addr learned my addr 172.21.15.64:0/1722557768 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.862+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.862+0000 7fcb64920700 1 -- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 msgr2=0x55fefb0c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.862+0000 7fcb64920700 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.862+0000 7fcb64920700 1 -- 172.21.15.64:0/1722557768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 msgr2=0x55fefb0c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.862+0000 7fcb64920700 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.862+0000 7fcb64920700 1 -- 172.21.15.64:0/1722557768 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fefae2ab00 con 0x55fefb0b5bf0 2024-05-08T21:59:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.863+0000 7fcb64920700 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0c1d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb48024f30 tx=0x7fcb48011fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d47e9d6e3afd3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.863+0000 7fcb3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.863+0000 7fcb3ffff700 1 -- 172.21.15.64:0/1722557768 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4801a600 con 0x55fefb0b5bf0 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.863+0000 7fcb3ffff700 1 -- 172.21.15.64:0/1722557768 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcb48029030 con 0x55fefb0b5bf0 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.863+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb5000beb0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.864+0000 7fcb3ffff700 1 -- 172.21.15.64:0/1722557768 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb4801a760 con 0x55fefb0b5bf0 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.864+0000 7fcb72d3eb80 10 client.25279 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.864+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fefae82c70 con 0x55fefb0b5bf0 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb3ffff700 1 -- 172.21.15.64:0/1722557768 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5000beb0 con 0x55fefb0b5bf0 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb3ffff700 1 -- 172.21.15.64:0/1722557768 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb4803e070 con 0x55fefb0b5bf0 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb72d3eb80 10 client.25279 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb72d3eb80 10 client.25279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb72d3eb80 20 client.25279 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0adf40 0x55fefb0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb72d3eb80 4 client.25279 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:59:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.865+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fefaff5e60 con 0x55fefb0adf40 2024-05-08T21:59:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.868+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0adf40 0x55fefb0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.868+0000 7fcb5ffff700 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0adf40 0x55fefb0b0360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a222d7d4b291db5 server_cookie=cec913b62b393935 in_seq=0 out_seq=0 2024-05-08T21:59:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.868+0000 7fcb3ffff700 10 client.25279 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb3ffff700 1 -- 172.21.15.64:0/1722557768 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fefaff5e60 con 0x55fefb0adf40 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb3ffff700 10 client.25279 handle_command_reply: tid=0 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 client.25279 shutdown 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 msgr2=0x55fefb0c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0c1d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb48024f30 tx=0x7fcb48011fb0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 shutdown_connections 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fefb0b6ae0 0x55fefb0c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55fefb0adf40 0x55fefb0b0360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.888+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fefb0b5290 0x55fefb0c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb72d3eb80 1 --2- 172.21.15.64:0/1722557768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fefb0b5bf0 0x55fefb0c1d40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 >> 172.21.15.64:0/1722557768 conn(0x55fefb00b1b0 msgr2=0x55fefb0bd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb3ffff700 0 client.25279 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 shutdown_connections 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb72d3eb80 1 -- 172.21.15.64:0/1722557768 wait complete. 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb72d3eb80 20 client.25279 trim_cache size 0 max 16384 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:59:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:59:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcb4c0a11a0 msgr2=0x7fcb4c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb66b84700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcb4c0a11a0 0x7fcb4c0a3660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb58094ed0 tx=0x7fcb5800cf40 comp rx=0 tx=0).stop 2024-05-08T21:59:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 msgr2=0x7fcb60100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.889+0000 7fcb66b84700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 0x7fcb60100680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800ef30 tx=0x7fcb5800c450 comp rx=0 tx=0).stop 2024-05-08T21:59:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 shutdown_connections 2024-05-08T21:59:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcb4c0a11a0 0x7fcb4c0a3660 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb60102b30 0x7fcb60100680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600ff1c0 0x7fcb60101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 --2- 172.21.15.64:0/1317891583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb60102190 0x7fcb60100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 >> 172.21.15.64:0/1317891583 conn(0x7fcb600faaf0 msgr2=0x7fcb60105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 shutdown_connections 2024-05-08T21:59:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:29.890+0000 7fcb66b84700 1 -- 172.21.15.64:0/1317891583 wait complete. 2024-05-08T21:59:29.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:29 smithi186 ceph-mon[46619]: pgmap v405: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 1.35k op/s 2024-05-08T21:59:29.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:59:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:30.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:59:30.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:59:30.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:31.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:59:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:59:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T21:59:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 280.0 s, 2790.0 tps, lat 35.731 ms stddev 43.560 2024-05-08T21:59:32.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:32 smithi064 ceph-mon[42500]: pgmap v406: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-05-08T21:59:32.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:32 smithi160 ceph-mon[46962]: pgmap v406: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-05-08T21:59:32.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:32 smithi186 ceph-mon[46619]: pgmap v406: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-05-08T21:59:34.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:34 smithi064 ceph-mon[42500]: pgmap v407: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 943 op/s 2024-05-08T21:59:34.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:34 smithi160 ceph-mon[46962]: pgmap v407: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 943 op/s 2024-05-08T21:59:34.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:34 smithi186 ceph-mon[46619]: pgmap v407: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 943 op/s 2024-05-08T21:59:36.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:36 smithi064 ceph-mon[42500]: pgmap v408: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 950 op/s 2024-05-08T21:59:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 285.0 s, 3394.1 tps, lat 29.426 ms stddev 33.657 2024-05-08T21:59:36.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:36 smithi160 ceph-mon[46962]: pgmap v408: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 950 op/s 2024-05-08T21:59:36.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:36 smithi186 ceph-mon[46619]: pgmap v408: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 950 op/s 2024-05-08T21:59:37.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:37 smithi064 ceph-mon[42500]: pgmap v409: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 815 op/s 2024-05-08T21:59:37.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:37 smithi160 ceph-mon[46962]: pgmap v409: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 815 op/s 2024-05-08T21:59:37.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:37 smithi186 ceph-mon[46619]: pgmap v409: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 815 op/s 2024-05-08T21:59:39.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:39 smithi186 ceph-mon[46619]: pgmap v410: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 800 op/s 2024-05-08T21:59:39.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:39 smithi064 ceph-mon[42500]: pgmap v410: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 800 op/s 2024-05-08T21:59:39.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:39 smithi160 ceph-mon[46962]: pgmap v410: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 800 op/s 2024-05-08T21:59:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 290.0 s, 2245.5 tps, lat 44.361 ms stddev 54.934 2024-05-08T21:59:41.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:41 smithi186 ceph-mon[46619]: pgmap v411: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 878 op/s 2024-05-08T21:59:41.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:41 smithi064 ceph-mon[42500]: pgmap v411: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 878 op/s 2024-05-08T21:59:41.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:41 smithi160 ceph-mon[46962]: pgmap v411: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 878 op/s 2024-05-08T21:59:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/2433372221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 msgr2=0x7fb2cc100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/2433372221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc100260 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009990 tx=0x7fb2bc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/2433372221 shutdown_connections 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/2433372221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2cc101950 0x7fb2cc108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/2433372221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc100260 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/2433372221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 0x7fb2cc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/2433372221 >> 172.21.15.64:0/2433372221 conn(0x7fb2cc0fb3b0 msgr2=0x7fb2cc0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/2433372221 shutdown_connections 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/2433372221 wait complete. 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.062+0000 7fb2d1c1e700 1 Processor -- start 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 -- start start 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 0x7fb2cc1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2cc101950 0x7fb2cc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc19e9e0 con 0x7fb2cc0ffe40 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc19eb20 con 0x7fb2cc101950 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2d1c1e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc19ecb0 con 0x7fb2cc100f90 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50822/0 (socket says 172.21.15.64:50822) 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/1674846871 learned_addr learned my addr 172.21.15.64:0/1674846871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2caffd700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 0x7fb2cc1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2cc101950 0x7fb2cc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2caffd700 1 -- 172.21.15.64:0/1674846871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2cc101950 msgr2=0x7fb2cc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2caffd700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2cc101950 0x7fb2cc19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2caffd700 1 -- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 msgr2=0x7fb2cc196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2caffd700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.063+0000 7fb2caffd700 1 -- 172.21.15.64:0/1674846871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0095f0 con 0x7fb2cc100f90 2024-05-08T21:59:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.064+0000 7fb2caffd700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 0x7fb2cc1973f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009ed0 tx=0x7fb2c000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.064+0000 7fb2c8ff9700 1 -- 172.21.15.64:0/1674846871 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0004220 con 0x7fb2cc100f90 2024-05-08T21:59:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.064+0000 7fb2c8ff9700 1 -- 172.21.15.64:0/1674846871 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c0004cf0 con 0x7fb2cc100f90 2024-05-08T21:59:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.064+0000 7fb2c8ff9700 1 -- 172.21.15.64:0/1674846871 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0013dc0 con 0x7fb2cc100f90 2024-05-08T21:59:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.064+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc1022a0 con 0x7fb2cc100f90 2024-05-08T21:59:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.064+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc102760 con 0x7fb2cc100f90 2024-05-08T21:59:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.067+0000 7fb2c8ff9700 1 -- 172.21.15.64:0/1674846871 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb2c0004380 con 0x7fb2cc100f90 2024-05-08T21:59:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.068+0000 7fb2dddd8b80 1 Processor -- start 2024-05-08T21:59:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.068+0000 7fb2dddd8b80 1 -- start start 2024-05-08T21:59:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdb290 0x5621efbdb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbdc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbdcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efb2a0f0 con 0x5621efbdbbf0 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efb27ab0 con 0x5621efbdb290 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efb233a0 con 0x5621efbdcae0 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbdc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbdc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50838/0 (socket says 172.21.15.64:50838) 2024-05-08T21:59:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/3963093318 learned_addr learned my addr 172.21.15.64:0/3963093318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2caffd700 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbdcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.069+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdb290 0x5621efbdb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2c8ff9700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb2b40a1220 0x7fb2b40a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/3963093318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 msgr2=0x5621efbdcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbdcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/3963093318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdb290 msgr2=0x5621efbdb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdb290 0x5621efbdb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/3963093318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ef9a4350 con 0x5621efbdbbf0 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdb290 0x5621efbdb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb2b40a1220 0x7fb2b40a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.070+0000 7fb2c8ff9700 1 -- 172.21.15.64:0/1674846871 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb2c00d0b30 con 0x7fb2cc100f90 2024-05-08T21:59:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.071+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbdc500 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005f30 tx=0x7fb2bc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab030cb07e62b920 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.072+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb2b40a1220 0x7fb2b40a36e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009ec0 tx=0x7fb2c4004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.073+0000 7fb2b1ffb700 1 -- 172.21.15.64:0/3963093318 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc01d070 con 0x5621efbdbbf0 2024-05-08T21:59:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.073+0000 7fb2b1ffb700 1 -- 172.21.15.64:0/3963093318 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2bc01e040 con 0x5621efbdbbf0 2024-05-08T21:59:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.073+0000 7fb2b1ffb700 1 -- 172.21.15.64:0/3963093318 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0049d0 con 0x5621efbdbbf0 2024-05-08T21:59:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/3963093318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 msgr2=0x5621efbdc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbdc500 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005f30 tx=0x7fb2bc005bc0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/3963093318 shutdown_connections 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdb290 0x5621efbdb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbdcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/3963093318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbdc500 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/3963093318 >> 172.21.15.64:0/3963093318 conn(0x5621efb311b0 msgr2=0x5621efbcf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/3963093318 shutdown_connections 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/3963093318 wait complete. 2024-05-08T21:59:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 -- start start 2024-05-08T21:59:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.074+0000 7fb2dddd8b80 1 Processor -- start 2024-05-08T21:59:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efbd3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efbd44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbd9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efabf500 con 0x5621efbdb290 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efabf1a0 con 0x5621efbdbbf0 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ef9a6640 con 0x5621efbdcae0 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2dddd8b80 10 client.0 _ll_register_callbacks cb 0x5621efac8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efbd3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efbd3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50852/0 (socket says 172.21.15.64:50852) 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/2988289433 learned_addr learned my addr 172.21.15.64:0/2988289433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbd9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2caffd700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efbd44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2caffd700 1 -- 172.21.15.64:0/2988289433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 msgr2=0x5621efbd9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2caffd700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbd9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2caffd700 1 -- 172.21.15.64:0/2988289433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 msgr2=0x5621efbd3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2caffd700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efbd3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.075+0000 7fb2caffd700 1 -- 172.21.15.64:0/2988289433 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c40132f0 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2caffd700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efbd44d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5621efbdc2c0 tx=0x7fb2c009e470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59f269fd0c88646 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2aaffd700 1 -- 172.21.15.64:0/2988289433 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00112e0 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2aaffd700 1 -- 172.21.15.64:0/2988289433 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb2c009e630 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621efa7c210 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2aaffd700 1 -- 172.21.15.64:0/2988289433 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c009da70 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 10 client.25385 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fb2cc108ba0 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2aaffd700 1 -- 172.21.15.64:0/2988289433 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621efa7c210 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2aaffd700 1 -- 172.21.15.64:0/2988289433 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2c00a6040 con 0x5621efbdbbf0 2024-05-08T21:59:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 10 client.25385 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 20 client.25385 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efbf9630 0x5621efbfba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 4 client.25385 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.076+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621efba27b0 con 0x5621efbf9630 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.077+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efbf9630 0x5621efbfba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.077+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efbf9630 0x5621efbfba20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e0753bf0dffcc5 server_cookie=389fe705b171971e in_seq=0 out_seq=0 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.077+0000 7fb2aaffd700 10 client.25385 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2aaffd700 1 -- 172.21.15.64:0/2988289433 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621efba27b0 con 0x5621efbf9630 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2aaffd700 10 client.25385 handle_command_reply: tid=0 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 client.25385 shutdown 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 msgr2=0x5621efbd44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efbd44d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5621efbdc2c0 tx=0x7fb2c009e470 comp rx=0 tx=0).stop 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 shutdown_connections 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efbf9630 0x5621efbfba20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efbd9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2aaffd700 0 client.25385 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efbd3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/2988289433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efbd44d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 >> 172.21.15.64:0/2988289433 conn(0x5621efb311b0 msgr2=0x5621efbcd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 shutdown_connections 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/2988289433 wait complete. 2024-05-08T21:59:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.078+0000 7fb2dddd8b80 20 client.25385 trim_cache size 0 max 16384 2024-05-08T21:59:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 Processor -- start 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 -- start start 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdb290 0x5621efbd2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbd34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 0x5621efbdcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efbc8330 con 0x5621efbdbbf0 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efbdc960 con 0x5621efbdcae0 2024-05-08T21:59:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.100+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efbd3b50 con 0x5621efbdb290 2024-05-08T21:59:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 0x5621efbdcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 0x5621efbdcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:55244/0 (socket says 172.21.15.64:55244) 2024-05-08T21:59:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2cbfff700 1 -- 172.21.15.64:0/1073523483 learned_addr learned my addr 172.21.15.64:0/1073523483 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdb290 0x5621efbd2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2caffd700 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbd34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2cbfff700 1 -- 172.21.15.64:0/1073523483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdb290 msgr2=0x5621efbd2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.101+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdb290 0x5621efbd2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2cbfff700 1 -- 172.21.15.64:0/1073523483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 msgr2=0x5621efbd34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbd34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2cbfff700 1 -- 172.21.15.64:0/1073523483 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ef950b00 con 0x5621efbdcae0 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2cbfff700 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 0x5621efbdcec0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4013780 tx=0x7fb2c4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e17db6de845eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2b27fc700 1 -- 172.21.15.64:0/1073523483 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c400e040 con 0x5621efbdcae0 2024-05-08T21:59:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2b27fc700 1 -- 172.21.15.64:0/1073523483 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c400f040 con 0x5621efbdcae0 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.102+0000 7fb2b27fc700 1 -- 172.21.15.64:0/1073523483 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4025b60 con 0x5621efbdcae0 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/1073523483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 msgr2=0x5621efbdcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 0x5621efbdcec0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4013780 tx=0x7fb2c4017000 comp rx=0 tx=0).stop 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/1073523483 shutdown_connections 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdcae0 0x5621efbdcec0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdb290 0x5621efbd2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/1073523483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdbbf0 0x5621efbd34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/1073523483 >> 172.21.15.64:0/1073523483 conn(0x5621efb311b0 msgr2=0x5621efbe30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.103+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/1073523483 shutdown_connections 2024-05-08T21:59:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.104+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/1073523483 wait complete. 2024-05-08T21:59:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.104+0000 7fb2dddd8b80 1 -- start start 2024-05-08T21:59:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.104+0000 7fb2dddd8b80 1 Processor -- start 2024-05-08T21:59:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efc078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efc09e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efc5f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efc5fb40 con 0x5621efbdb290 2024-05-08T21:59:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efc5fcb0 con 0x5621efbdbbf0 2024-05-08T21:59:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621efc5fe70 con 0x5621efbdcae0 2024-05-08T21:59:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.115+0000 7fb2dddd8b80 10 client.0 _ll_register_callbacks cb 0x5621efac8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.121+0000 7fb2cbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efc5f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.121+0000 7fb2cbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efc5f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:55150/0 (socket says 172.21.15.64:55150) 2024-05-08T21:59:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.121+0000 7fb2cbfff700 1 -- 172.21.15.64:0/42413546 learned_addr learned my addr 172.21.15.64:0/42413546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.121+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efc078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/42413546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 msgr2=0x5621efc5f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efc5f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 msgr2=0x5621efc09e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efc09e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2cb7fe700 1 -- 172.21.15.64:0/42413546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c4013480 con 0x5621efbdb290 2024-05-08T21:59:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2cb7fe700 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efc078d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009ce0 tx=0x7fb2bc005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff49cf00906ef0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T21:59:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2aaffd700 1 -- 172.21.15.64:0/42413546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc01d040 con 0x5621efbdb290 2024-05-08T21:59:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.122+0000 7fb2aaffd700 1 -- 172.21.15.64:0/42413546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb2bc00baa0 con 0x5621efbdb290 2024-05-08T21:59:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.123+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2bc01e040 con 0x5621efbdb290 2024-05-08T21:59:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.123+0000 7fb2aaffd700 1 -- 172.21.15.64:0/42413546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2bc00bc60 con 0x5621efbdb290 2024-05-08T21:59:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.125+0000 7fb2dddd8b80 10 client.15654 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.125+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5621efae9500 con 0x5621efbdb290 2024-05-08T21:59:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2aaffd700 1 -- 172.21.15.64:0/42413546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2bc01e040 con 0x5621efbdb290 2024-05-08T21:59:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2aaffd700 1 -- 172.21.15.64:0/42413546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2bc0045d0 con 0x5621efbdb290 2024-05-08T21:59:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2dddd8b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2dddd8b80 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2dddd8b80 20 client.15654 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efc613a0 0x5621efc63790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2dddd8b80 4 client.15654 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:59:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.126+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5621efb1be60 con 0x5621efc613a0 2024-05-08T21:59:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.129+0000 7fb2caffd700 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efc613a0 0x5621efc63790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.136+0000 7fb2caffd700 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efc613a0 0x5621efc63790 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cd652fc6272af8 server_cookie=9f8db742a3c0b193 in_seq=0 out_seq=0 2024-05-08T21:59:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.136+0000 7fb2aaffd700 10 client.15654 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.165+0000 7fb2aaffd700 1 -- 172.21.15.64:0/42413546 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5621efb1be60 con 0x5621efc613a0 2024-05-08T21:59:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.165+0000 7fb2aaffd700 10 client.15654 handle_command_reply: tid=0 2024-05-08T21:59:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.165+0000 7fb2dddd8b80 1 client.15654 shutdown 2024-05-08T21:59:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.168+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 msgr2=0x5621efc078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.168+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efc078d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009ce0 tx=0x7fb2bc005500 comp rx=0 tx=0).stop 2024-05-08T21:59:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 shutdown_connections 2024-05-08T21:59:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5621efc613a0 0x5621efc63790 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621efbdcae0 0x5621efc5f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621efbdb290 0x5621efc078d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 --2- 172.21.15.64:0/42413546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5621efbdbbf0 0x5621efc09e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2aaffd700 0 client.15654 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 >> 172.21.15.64:0/42413546 conn(0x5621efb311b0 msgr2=0x5621efbcf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.172+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 shutdown_connections 2024-05-08T21:59:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.173+0000 7fb2dddd8b80 1 -- 172.21.15.64:0/42413546 wait complete. 2024-05-08T21:59:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2dddd8b80 20 client.15654 trim_cache size 0 max 16384 2024-05-08T21:59:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:59:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:59:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:59:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:59:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb2b40a1220 msgr2=0x7fb2b40a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb2b40a1220 0x7fb2b40a36e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009ec0 tx=0x7fb2c4004ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 msgr2=0x7fb2cc1973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 0x7fb2cc1973f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009ed0 tx=0x7fb2c000ecd0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 shutdown_connections 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb2b40a1220 0x7fb2b40a36e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2cc101950 0x7fb2cc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ffe40 0x7fb2cc196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 --2- 172.21.15.64:0/1674846871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2cc100f90 0x7fb2cc1973f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.174+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 >> 172.21.15.64:0/1674846871 conn(0x7fb2cc0fb3b0 msgr2=0x7fb2cc069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.175+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 shutdown_connections 2024-05-08T21:59:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:43.175+0000 7fb2d1c1e700 1 -- 172.21.15.64:0/1674846871 wait complete. 2024-05-08T21:59:43.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:43 smithi186 ceph-mon[46619]: pgmap v412: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 823 op/s 2024-05-08T21:59:43.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:43 smithi064 ceph-mon[42500]: pgmap v412: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 823 op/s 2024-05-08T21:59:43.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:43 smithi160 ceph-mon[46962]: pgmap v412: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 823 op/s 2024-05-08T21:59:45.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:45 smithi186 ceph-mon[46619]: pgmap v413: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 902 op/s 2024-05-08T21:59:45.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:45 smithi064 ceph-mon[42500]: pgmap v413: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 902 op/s 2024-05-08T21:59:45.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:45 smithi160 ceph-mon[46962]: pgmap v413: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 902 op/s 2024-05-08T21:59:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 295.0 s, 2033.0 tps, lat 49.352 ms stddev 63.890 2024-05-08T21:59:47.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:47 smithi064 ceph-mon[42500]: pgmap v414: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 826 op/s 2024-05-08T21:59:47.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:47 smithi160 ceph-mon[46962]: pgmap v414: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 826 op/s 2024-05-08T21:59:47.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:47 smithi186 ceph-mon[46619]: pgmap v414: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 826 op/s 2024-05-08T21:59:49.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:49 smithi064 ceph-mon[42500]: pgmap v415: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 852 op/s 2024-05-08T21:59:49.798 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:49 smithi160 ceph-mon[46962]: pgmap v415: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 852 op/s 2024-05-08T21:59:49.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:49 smithi186 ceph-mon[46619]: pgmap v415: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 852 op/s 2024-05-08T21:59:51.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 300.0 s, 2251.0 tps, lat 44.174 ms stddev 52.003 2024-05-08T21:59:51.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:51 smithi064 ceph-mon[42500]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 910 op/s 2024-05-08T21:59:51.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:51 smithi160 ceph-mon[46962]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 910 op/s 2024-05-08T21:59:51.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:51 smithi186 ceph-mon[46619]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 910 op/s 2024-05-08T21:59:53.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:53 smithi064 ceph-mon[42500]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 829 op/s 2024-05-08T21:59:53.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:53 smithi160 ceph-mon[46962]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 829 op/s 2024-05-08T21:59:53.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:53 smithi186 ceph-mon[46619]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 829 op/s 2024-05-08T21:59:55.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:55 smithi064 ceph-mon[42500]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 898 op/s 2024-05-08T21:59:55.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:55 smithi160 ceph-mon[46962]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 898 op/s 2024-05-08T21:59:55.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:55 smithi186 ceph-mon[46619]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 898 op/s 2024-05-08T21:59:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.358+0000 7fbcacf59700 1 -- 172.21.15.64:0/1618601069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 msgr2=0x7fbca810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.358+0000 7fbcacf59700 1 --2- 172.21.15.64:0/1618601069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca810a3d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009990 tx=0x7fbc98009ca0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 -- 172.21.15.64:0/1618601069 shutdown_connections 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 --2- 172.21.15.64:0/1618601069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810d2e0 0x7fbca810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 --2- 172.21.15.64:0/1618601069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca810a3d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 --2- 172.21.15.64:0/1618601069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca810a910 0x7fbca810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 -- 172.21.15.64:0/1618601069 >> 172.21.15.64:0/1618601069 conn(0x7fbca806df30 msgr2=0x7fbca8070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 -- 172.21.15.64:0/1618601069 shutdown_connections 2024-05-08T21:59:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.369+0000 7fbcacf59700 1 -- 172.21.15.64:0/1618601069 wait complete. 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 Processor -- start 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 -- start start 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca81a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca810a910 0x7fbca81a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810d2e0 0x7fbca81ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca810f6d0 con 0x7fbca8107fa0 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca810f840 con 0x7fbca810d2e0 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.370+0000 7fbcacf59700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca810fa00 con 0x7fbca810a910 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca810a910 0x7fbca81a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca81a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca6d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810d2e0 0x7fbca81ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca81a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37424/0 (socket says 172.21.15.64:37424) 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 -- 172.21.15.64:0/4107900803 learned_addr learned my addr 172.21.15.64:0/4107900803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 -- 172.21.15.64:0/4107900803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca810a910 msgr2=0x7fbca81a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca810a910 0x7fbca81a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 -- 172.21.15.64:0/4107900803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810d2e0 msgr2=0x7fbca81ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810d2e0 0x7fbca81ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 -- 172.21.15.64:0/4107900803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc980095f0 con 0x7fbca8107fa0 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbca659c700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca81a5cf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b790 tx=0x7fbc9800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.371+0000 7fbc977fe700 1 -- 172.21.15.64:0/4107900803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9801d070 con 0x7fbca8107fa0 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.372+0000 7fbc977fe700 1 -- 172.21.15.64:0/4107900803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9800fc20 con 0x7fbca8107fa0 2024-05-08T21:59:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.372+0000 7fbc977fe700 1 -- 172.21.15.64:0/4107900803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98017650 con 0x7fbca8107fa0 2024-05-08T21:59:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.374+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca810fc80 con 0x7fbca8107fa0 2024-05-08T21:59:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.374+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca81100b0 con 0x7fbca8107fa0 2024-05-08T21:59:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.377+0000 7fbcb9113b80 1 Processor -- start 2024-05-08T21:59:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 -- start start 2024-05-08T21:59:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec16a80 0x55dcbec16ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec17cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec18750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbeb6e0f0 con 0x55dcbec182d0 2024-05-08T21:59:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbeb6bab0 con 0x55dcbec16a80 2024-05-08T21:59:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbeb673a0 con 0x55dcbec173e0 2024-05-08T21:59:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbca659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec17cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbca659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec17cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56958/0 (socket says 172.21.15.64:56958) 2024-05-08T21:59:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbca659c700 1 -- 172.21.15.64:0/314611747 learned_addr learned my addr 172.21.15.64:0/314611747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.378+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec16a80 0x55dcbec16ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.379+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec18750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.379+0000 7fbca6d9d700 1 -- 172.21.15.64:0/314611747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 msgr2=0x55dcbec17cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.379+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec17cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.379+0000 7fbca6d9d700 1 -- 172.21.15.64:0/314611747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 msgr2=0x55dcbec18750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.379+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec18750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.379+0000 7fbca6d9d700 1 -- 172.21.15.64:0/314611747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcbe9e8350 con 0x55dcbec16a80 2024-05-08T21:59:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.380+0000 7fbc977fe700 1 -- 172.21.15.64:0/4107900803 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbc98021410 con 0x7fbca8107fa0 2024-05-08T21:59:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.381+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec16a80 0x55dcbec16ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009310 tx=0x7fbca0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda84b69b1407e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.381+0000 7fbc977fe700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbc900a1170 0x7fbc900a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.381+0000 7fbc977fe700 1 -- 172.21.15.64:0/4107900803 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc980d4ad0 con 0x7fbca8107fa0 2024-05-08T21:59:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbc94ff9700 1 -- 172.21.15.64:0/314611747 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000e070 con 0x55dcbec16a80 2024-05-08T21:59:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbc94ff9700 1 -- 172.21.15.64:0/314611747 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca000f040 con 0x55dcbec16a80 2024-05-08T21:59:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbc94ff9700 1 -- 172.21.15.64:0/314611747 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0014840 con 0x55dcbec16a80 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 -- 172.21.15.64:0/314611747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec16a80 msgr2=0x55dcbec16ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec16a80 0x55dcbec16ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009310 tx=0x7fbca0009920 comp rx=0 tx=0).stop 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 -- 172.21.15.64:0/314611747 shutdown_connections 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec16a80 0x55dcbec16ea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec18750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/314611747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec17cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbca659c700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbc900a1170 0x7fbc900a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.385+0000 7fbcb9113b80 1 -- 172.21.15.64:0/314611747 >> 172.21.15.64:0/314611747 conn(0x55dcbeb751b0 msgr2=0x55dcbec13440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.386+0000 7fbcb9113b80 1 -- 172.21.15.64:0/314611747 shutdown_connections 2024-05-08T21:59:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.387+0000 7fbcb9113b80 1 -- 172.21.15.64:0/314611747 wait complete. 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.387+0000 7fbcb9113b80 1 -- start start 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.388+0000 7fbcb9113b80 1 Processor -- start 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec16a80 0x55dcbec20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 0x55dcbec20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec27ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbeb03500 con 0x55dcbec182d0 2024-05-08T21:59:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbeb031a0 con 0x55dcbec173e0 2024-05-08T21:59:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbe9ea640 con 0x55dcbec16a80 2024-05-08T21:59:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.389+0000 7fbcb9113b80 10 client.0 _ll_register_callbacks cb 0x55dcbeb0cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 0x55dcbec20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca659c700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbc900a1170 0x7fbc900a3630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc980a1bd0 tx=0x7fbc98058040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 0x55dcbec20e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54318/0 (socket says 172.21.15.64:54318) 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 -- 172.21.15.64:0/3809290421 learned_addr learned my addr 172.21.15.64:0/3809290421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 -- 172.21.15.64:0/3809290421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec16a80 msgr2=0x55dcbec20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec16a80 0x55dcbec20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 -- 172.21.15.64:0/3809290421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 msgr2=0x55dcbec27ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec27ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.390+0000 7fbca5d9b700 1 -- 172.21.15.64:0/3809290421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca0009050 con 0x55dcbec173e0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 0x55dcbec20e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f210 tx=0x7fbc9c00f5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b2141843c4c0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbc8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbc8dffb700 1 -- 172.21.15.64:0/3809290421 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c014660 con 0x55dcbec173e0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbc8dffb700 1 -- 172.21.15.64:0/3809290421 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc9c0147c0 con 0x55dcbec173e0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcbeac0210 con 0x55dcbec173e0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbc8dffb700 1 -- 172.21.15.64:0/3809290421 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc9c014980 con 0x55dcbec173e0 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbcb9113b80 10 client.25400 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.391+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55dcbe994b00 con 0x55dcbec173e0 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbc8dffb700 1 -- 172.21.15.64:0/3809290421 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcbeac0210 con 0x55dcbec173e0 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbc8dffb700 1 -- 172.21.15.64:0/3809290421 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc9c01fc10 con 0x55dcbec173e0 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbcb9113b80 10 client.25400 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbcb9113b80 10 client.25400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbcb9113b80 20 client.25400 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec2c830 0x55dcbec2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbcb9113b80 4 client.25400 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:59:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.392+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcbebe67b0 con 0x55dcbec2c830 2024-05-08T21:59:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.396+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec2c830 0x55dcbec2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.398+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec2c830 0x55dcbec2ecb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2bcf921be5788a server_cookie=2238764848d9af36 in_seq=0 out_seq=0 2024-05-08T21:59:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.398+0000 7fbc8dffb700 10 client.25400 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbc8dffb700 1 -- 172.21.15.64:0/3809290421 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcbebe67b0 con 0x55dcbec2c830 2024-05-08T21:59:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbc8dffb700 10 client.25400 handle_command_reply: tid=0 2024-05-08T21:59:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 client.25400 shutdown 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 msgr2=0x55dcbec20e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 0x55dcbec20e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f210 tx=0x7fbc9c00f5d0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 shutdown_connections 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec2c830 0x55dcbec2ecb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec182d0 0x55dcbec27ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec16a80 0x55dcbec20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/3809290421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec173e0 0x55dcbec20e60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 >> 172.21.15.64:0/3809290421 conn(0x55dcbeb751b0 msgr2=0x55dcbec11860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.399+0000 7fbc8dffb700 0 client.25400 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.401+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 shutdown_connections 2024-05-08T21:59:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.401+0000 7fbcb9113b80 1 -- 172.21.15.64:0/3809290421 wait complete. 2024-05-08T21:59:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.401+0000 7fbcb9113b80 20 client.25400 trim_cache size 0 max 16384 2024-05-08T21:59:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.426+0000 7fbcb9113b80 1 Processor -- start 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.426+0000 7fbcb9113b80 1 -- start start 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.426+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec1f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.426+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.426+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbec1e550 con 0x55dcbec16a80 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbec1be60 con 0x55dcbec182d0 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbec2c5b0 con 0x55dcbec173e0 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec1f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec1f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37448/0 (socket says 172.21.15.64:37448) 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 -- 172.21.15.64:0/4019525631 learned_addr learned my addr 172.21.15.64:0/4019525631 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 -- 172.21.15.64:0/4019525631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 msgr2=0x55dcbec28550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec28550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 -- 172.21.15.64:0/4019525631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 msgr2=0x55dcbec1f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec1f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.427+0000 7fbca6d9d700 1 -- 172.21.15.64:0/4019525631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcbeb2d500 con 0x55dcbec182d0 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.428+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec186b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbca0015f70 tx=0x7fbca0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2e3b83add3d9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.428+0000 7fbc957fa700 1 -- 172.21.15.64:0/4019525631 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca002b040 con 0x55dcbec182d0 2024-05-08T21:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.428+0000 7fbc957fa700 1 -- 172.21.15.64:0/4019525631 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca002c040 con 0x55dcbec182d0 2024-05-08T21:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.428+0000 7fbc957fa700 1 -- 172.21.15.64:0/4019525631 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0024040 con 0x55dcbec182d0 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 -- 172.21.15.64:0/4019525631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 msgr2=0x55dcbec186b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec186b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbca0015f70 tx=0x7fbca0000f40 comp rx=0 tx=0).stop 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 -- 172.21.15.64:0/4019525631 shutdown_connections 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec186b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec1f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/4019525631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec28550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.429+0000 7fbcb9113b80 1 -- 172.21.15.64:0/4019525631 >> 172.21.15.64:0/4019525631 conn(0x55dcbeb751b0 msgr2=0x55dcbec1fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.430+0000 7fbcb9113b80 1 -- 172.21.15.64:0/4019525631 shutdown_connections 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.430+0000 7fbcb9113b80 1 -- 172.21.15.64:0/4019525631 wait complete. 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.430+0000 7fbcb9113b80 1 -- start start 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.430+0000 7fbcb9113b80 1 Processor -- start 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec25ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec26a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbec26f70 con 0x55dcbec16a80 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbec270e0 con 0x55dcbec182d0 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbec272a0 con 0x55dcbec173e0 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbcb9113b80 10 client.0 _ll_register_callbacks cb 0x55dcbeb0cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec26a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec26a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54336/0 (socket says 172.21.15.64:54336) 2024-05-08T21:59:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 -- 172.21.15.64:0/415993324 learned_addr learned my addr 172.21.15.64:0/415993324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec25ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 -- 172.21.15.64:0/415993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 msgr2=0x55dcbec25ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec25ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 -- 172.21.15.64:0/415993324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 msgr2=0x55dcbec25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.431+0000 7fbca6d9d700 1 -- 172.21.15.64:0/415993324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcbe9e8350 con 0x55dcbec182d0 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.432+0000 7fbca6d9d700 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec26a30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbca0007b30 tx=0x7fbca0016fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ee3b498133caf55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.432+0000 7fbc8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T21:59:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.433+0000 7fbc8dffb700 1 -- 172.21.15.64:0/415993324 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca002b070 con 0x55dcbec182d0 2024-05-08T21:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.433+0000 7fbc8dffb700 1 -- 172.21.15.64:0/415993324 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbca0026580 con 0x55dcbec182d0 2024-05-08T21:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.433+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbca000f040 con 0x55dcbec182d0 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbc8dffb700 1 -- 172.21.15.64:0/415993324 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca000e040 con 0x55dcbec182d0 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 10 client.25406 fetch_fsmap learned FSMap version 39 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55dcbe9ecc70 con 0x55dcbec182d0 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbc8dffb700 1 -- 172.21.15.64:0/415993324 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca000f040 con 0x55dcbec182d0 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbc8dffb700 1 -- 172.21.15.64:0/415993324 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbca0015440 con 0x55dcbec182d0 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 10 client.25406 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 20 client.25406 populate_metadata read hostname 'smithi064' 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec831a0 0x55dcbec855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 4 client.25406 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T21:59:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.436+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dcbeb5fe60 con 0x55dcbec831a0 2024-05-08T21:59:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.437+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec831a0 0x55dcbec855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.437+0000 7fbc8dffb700 10 client.25406 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.437+0000 7fbca5d9b700 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec831a0 0x55dcbec855c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d9115cf2a2f569 server_cookie=b8496d82d8467b17 in_seq=0 out_seq=0 2024-05-08T21:59:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbc8dffb700 1 -- 172.21.15.64:0/415993324 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dcbeb5fe60 con 0x55dcbec831a0 2024-05-08T21:59:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbc8dffb700 10 client.25406 handle_command_reply: tid=0 2024-05-08T21:59:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 client.25406 shutdown 2024-05-08T21:59:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 msgr2=0x55dcbec26a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec26a30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbca0007b30 tx=0x7fbca0016fb0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 shutdown_connections 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcbec182d0 0x55dcbec26a30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55dcbec831a0 0x55dcbec855c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcbec16a80 0x55dcbec25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 --2- 172.21.15.64:0/415993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dcbec173e0 0x55dcbec25ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 >> 172.21.15.64:0/415993324 conn(0x55dcbeb751b0 msgr2=0x55dcbec1b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.456+0000 7fbc8dffb700 0 client.25406 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T21:59:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.457+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 shutdown_connections 2024-05-08T21:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.460+0000 7fbcb9113b80 1 -- 172.21.15.64:0/415993324 wait complete. 2024-05-08T21:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.460+0000 7fbcb9113b80 20 client.25406 trim_cache size 0 max 16384 2024-05-08T21:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T21:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T21:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T21:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbc900a1170 msgr2=0x7fbc900a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbc900a1170 0x7fbc900a3630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc980a1bd0 tx=0x7fbc98058040 comp rx=0 tx=0).stop 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 msgr2=0x7fbca81a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca81a5cf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b790 tx=0x7fbc9800f510 comp rx=0 tx=0).stop 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 shutdown_connections 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbc900a1170 0x7fbc900a3630 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810d2e0 0x7fbca81ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbca8107fa0 0x7fbca81a5cf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 --2- 172.21.15.64:0/4107900803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbca810a910 0x7fbca81a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 >> 172.21.15.64:0/4107900803 conn(0x7fbca806df30 msgr2=0x7fbca810bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 shutdown_connections 2024-05-08T21:59:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T21:59:56.461+0000 7fbcacf59700 1 -- 172.21.15.64:0/4107900803 wait complete. 2024-05-08T21:59:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 305.0 s, 1519.4 tps, lat 65.691 ms stddev 99.011 2024-05-08T21:59:57.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:57 smithi064 ceph-mon[42500]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 884 op/s 2024-05-08T21:59:57.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:57 smithi160 ceph-mon[46962]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 884 op/s 2024-05-08T21:59:57.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:57 smithi186 ceph-mon[46619]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 884 op/s 2024-05-08T21:59:59.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 21:59:59 smithi064 ceph-mon[42500]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2024-05-08T21:59:59.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 21:59:59 smithi160 ceph-mon[46962]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2024-05-08T21:59:59.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 21:59:59 smithi186 ceph-mon[46619]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2024-05-08T22:00:00.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:00 smithi064 ceph-mon[42500]: overall HEALTH_OK 2024-05-08T22:00:00.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:00 smithi160 ceph-mon[46962]: overall HEALTH_OK 2024-05-08T22:00:00.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:00 smithi186 ceph-mon[46619]: overall HEALTH_OK 2024-05-08T22:00:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 310.0 s, 2053.2 tps, lat 49.278 ms stddev 56.612 2024-05-08T22:00:01.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:01 smithi064 ceph-mon[42500]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 1.98k op/s 2024-05-08T22:00:01.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:01 smithi160 ceph-mon[46962]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 1.98k op/s 2024-05-08T22:00:01.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:01 smithi186 ceph-mon[46619]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 1.98k op/s 2024-05-08T22:00:03.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:03 smithi064 ceph-mon[42500]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.94k op/s 2024-05-08T22:00:03.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:03 smithi160 ceph-mon[46962]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.94k op/s 2024-05-08T22:00:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:03 smithi186 ceph-mon[46619]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.94k op/s 2024-05-08T22:00:05.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:05 smithi064 ceph-mon[42500]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 2.01k op/s 2024-05-08T22:00:05.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:05 smithi160 ceph-mon[46962]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 2.01k op/s 2024-05-08T22:00:05.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:05 smithi186 ceph-mon[46619]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 2.01k op/s 2024-05-08T22:00:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 315.0 s, 2524.0 tps, lat 39.643 ms stddev 45.161 2024-05-08T22:00:07.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:07 smithi160 ceph-mon[46962]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2024-05-08T22:00:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:07 smithi186 ceph-mon[46619]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2024-05-08T22:00:07.973 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:07 smithi064 ceph-mon[42500]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2024-05-08T22:00:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 -- 172.21.15.64:0/3471748628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 msgr2=0x7fdf40102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 --2- 172.21.15.64:0/3471748628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 0x7fdf40102500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009990 tx=0x7fdf34009ca0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 -- 172.21.15.64:0/3471748628 shutdown_connections 2024-05-08T22:00:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 --2- 172.21.15.64:0/3471748628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 0x7fdf40109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 --2- 172.21.15.64:0/3471748628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf40101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 --2- 172.21.15.64:0/3471748628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 0x7fdf40102500 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 -- 172.21.15.64:0/3471748628 >> 172.21.15.64:0/3471748628 conn(0x7fdf400fc4a0 msgr2=0x7fdf400fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.633+0000 7fdf457e5700 1 -- 172.21.15.64:0/3471748628 shutdown_connections 2024-05-08T22:00:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.634+0000 7fdf457e5700 1 -- 172.21.15.64:0/3471748628 wait complete. 2024-05-08T22:00:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 Processor -- start 2024-05-08T22:00:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 -- start start 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf4006f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 0x7fdf4006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 0x7fdf401a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40072e30 con 0x7fdf40102080 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40072fa0 con 0x7fdf40102a40 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf457e5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf401a7500 con 0x7fdf40100f30 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf4006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf4006f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:35656/0 (socket says 172.21.15.64:35656) 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.635+0000 7fdf3effd700 1 -- 172.21.15.64:0/801631148 learned_addr learned my addr 172.21.15.64:0/801631148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 0x7fdf401a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 0x7fdf4006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3effd700 1 -- 172.21.15.64:0/801631148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 msgr2=0x7fdf401a6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3effd700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 0x7fdf401a6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3effd700 1 -- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 msgr2=0x7fdf4006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3effd700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 0x7fdf4006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3effd700 1 -- 172.21.15.64:0/801631148 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340095f0 con 0x7fdf40100f30 2024-05-08T22:00:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.636+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 0x7fdf401a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.638+0000 7fdf3effd700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf4006f860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000e9a0 tx=0x7fdf3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.638+0000 7fdf27fff700 1 -- 172.21.15.64:0/801631148 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009790 con 0x7fdf40100f30 2024-05-08T22:00:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.638+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf401a7700 con 0x7fdf40100f30 2024-05-08T22:00:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.638+0000 7fdf27fff700 1 -- 172.21.15.64:0/801631148 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3000cbd0 con 0x7fdf40100f30 2024-05-08T22:00:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.638+0000 7fdf27fff700 1 -- 172.21.15.64:0/801631148 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30018680 con 0x7fdf40100f30 2024-05-08T22:00:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.638+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf401a7bd0 con 0x7fdf40100f30 2024-05-08T22:00:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.641+0000 7fdf27fff700 1 -- 172.21.15.64:0/801631148 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdf3000cd40 con 0x7fdf40100f30 2024-05-08T22:00:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.642+0000 7fdf5199fb80 1 Processor -- start 2024-05-08T22:00:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.642+0000 7fdf27fff700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fdf280a11a0 0x7fdf280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.643+0000 7fdf27fff700 1 -- 172.21.15.64:0/801631148 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf30014070 con 0x7fdf40100f30 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.643+0000 7fdf5199fb80 1 -- start start 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fdf280a11a0 0x7fdf280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 0x55b249e85ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e86cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e872d0 0x55b249e87750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249ddd0f0 con 0x55b249e863e0 2024-05-08T22:00:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249ddaab0 con 0x55b249e872d0 2024-05-08T22:00:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249dd63a0 con 0x55b249e85a80 2024-05-08T22:00:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 0x55b249e85ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 0x55b249e85ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:35672/0 (socket says 172.21.15.64:35672) 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 -- 172.21.15.64:0/1047897750 learned_addr learned my addr 172.21.15.64:0/1047897750 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e872d0 0x55b249e87750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 -- 172.21.15.64:0/1047897750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e872d0 msgr2=0x55b249e87750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e872d0 0x55b249e87750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 -- 172.21.15.64:0/1047897750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 msgr2=0x55b249e86cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e86cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 -- 172.21.15.64:0/1047897750 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b249c57350 con 0x55b249e85a80 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.644+0000 7fdf3effd700 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 0x55b249e85ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf30099f00 tx=0x7fdf30099950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30431d9fcb90250e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.645+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fdf280a11a0 0x7fdf280a3660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf38003ee0 tx=0x7fdf38004320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.645+0000 7fdf257fa700 1 -- 172.21.15.64:0/1047897750 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30099f70 con 0x55b249e85a80 2024-05-08T22:00:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.645+0000 7fdf257fa700 1 -- 172.21.15.64:0/1047897750 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3009dda0 con 0x55b249e85a80 2024-05-08T22:00:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.645+0000 7fdf257fa700 1 -- 172.21.15.64:0/1047897750 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300a1f60 con 0x55b249e85a80 2024-05-08T22:00:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.645+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1047897750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 msgr2=0x55b249e85ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.645+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 0x55b249e85ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf30099f00 tx=0x7fdf30099950 comp rx=0 tx=0).stop 2024-05-08T22:00:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1047897750 shutdown_connections 2024-05-08T22:00:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e872d0 0x55b249e87750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e85a80 0x55b249e85ea0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1047897750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e86cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1047897750 >> 172.21.15.64:0/1047897750 conn(0x55b249de41b0 msgr2=0x55b249e82440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1047897750 shutdown_connections 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.647+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1047897750 wait complete. 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.648+0000 7fdf5199fb80 1 -- start start 2024-05-08T22:00:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.648+0000 7fdf5199fb80 1 Processor -- start 2024-05-08T22:00:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e980d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249d72500 con 0x55b249e863e0 2024-05-08T22:00:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249d721a0 con 0x55b249e85a80 2024-05-08T22:00:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249c59640 con 0x55b249e872d0 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf5199fb80 10 client.0 _ll_register_callbacks cb 0x55b249d7bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:42552/0 (socket says 172.21.15.64:42552) 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 -- 172.21.15.64:0/3087492804 learned_addr learned my addr 172.21.15.64:0/3087492804 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 -- 172.21.15.64:0/3087492804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 msgr2=0x55b249e9f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 -- 172.21.15.64:0/3087492804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 msgr2=0x55b249e98610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e98610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.649+0000 7fdf3effd700 1 -- 172.21.15.64:0/3087492804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf401a7bd0 con 0x55b249e85a80 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.650+0000 7fdf3effd700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e980d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf300a1ec0 tx=0x7fdf300962b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=766a827a3b90289f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.650+0000 7fdf22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:00:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.650+0000 7fdf22ff5700 1 -- 172.21.15.64:0/3087492804 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30098310 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.650+0000 7fdf22ff5700 1 -- 172.21.15.64:0/3087492804 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf300ade40 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.650+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b249d2f210 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.651+0000 7fdf22ff5700 1 -- 172.21.15.64:0/3087492804 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf30099f70 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.651+0000 7fdf5199fb80 10 client.25415 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.651+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b249c03b00 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.651+0000 7fdf22ff5700 1 -- 172.21.15.64:0/3087492804 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b249d2f210 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf22ff5700 1 -- 172.21.15.64:0/3087492804 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf300b2b20 con 0x55b249e85a80 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf5199fb80 10 client.25415 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf5199fb80 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf5199fb80 20 client.25415 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249e924f0 0x55b249e94910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf5199fb80 4 client.25415 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:00:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b249e557b0 con 0x55b249e924f0 2024-05-08T22:00:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.652+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249e924f0 0x55b249e94910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.653+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249e924f0 0x55b249e94910 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e79e1cbcdc86289 server_cookie=79ab60e5de2137f in_seq=0 out_seq=0 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.653+0000 7fdf22ff5700 10 client.25415 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.653+0000 7fdf22ff5700 1 -- 172.21.15.64:0/3087492804 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b249e557b0 con 0x55b249e924f0 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.653+0000 7fdf22ff5700 10 client.25415 handle_command_reply: tid=0 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.653+0000 7fdf5199fb80 1 client.25415 shutdown 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 msgr2=0x55b249e980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e980d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf300a1ec0 tx=0x7fdf300962b0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 shutdown_connections 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e980d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249e924f0 0x55b249e94910 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/3087492804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e863e0 0x55b249e98610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf22ff5700 0 client.25415 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 >> 172.21.15.64:0/3087492804 conn(0x55b249de41b0 msgr2=0x55b249e80860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 shutdown_connections 2024-05-08T22:00:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 1 -- 172.21.15.64:0/3087492804 wait complete. 2024-05-08T22:00:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.654+0000 7fdf5199fb80 20 client.25415 trim_cache size 0 max 16384 2024-05-08T22:00:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.674+0000 7fdf5199fb80 1 Processor -- start 2024-05-08T22:00:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 -- start start 2024-05-08T22:00:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e863e0 0x55b249e9f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e872d0 0x55b249e876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249e87150 con 0x55b249e872d0 2024-05-08T22:00:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249e8f1d0 con 0x55b249e85a80 2024-05-08T22:00:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249e81930 con 0x55b249e863e0 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e872d0 0x55b249e876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:42556/0 (socket says 172.21.15.64:42556) 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf3effd700 1 -- 172.21.15.64:0/1352072956 learned_addr learned my addr 172.21.15.64:0/1352072956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e872d0 0x55b249e876b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51696/0 (socket says 172.21.15.64:51696) 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.675+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e863e0 0x55b249e9f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1352072956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 msgr2=0x55b249e948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1352072956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e872d0 msgr2=0x55b249e876b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e872d0 0x55b249e876b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1352072956 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b249d9c500 con 0x55b249e863e0 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e863e0 0x55b249e9f610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400b620 tx=0x7fdf34005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1289d643647cceae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf25ffb700 1 -- 172.21.15.64:0/1352072956 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400b7f0 con 0x55b249e863e0 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf25ffb700 1 -- 172.21.15.64:0/1352072956 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3401e040 con 0x55b249e863e0 2024-05-08T22:00:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.676+0000 7fdf25ffb700 1 -- 172.21.15.64:0/1352072956 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34026840 con 0x55b249e863e0 2024-05-08T22:00:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1352072956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e863e0 msgr2=0x55b249e9f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e863e0 0x55b249e9f610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400b620 tx=0x7fdf34005a90 comp rx=0 tx=0).stop 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1352072956 shutdown_connections 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e85a80 0x55b249e948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e872d0 0x55b249e876b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1352072956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e863e0 0x55b249e9f610 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1352072956 >> 172.21.15.64:0/1352072956 conn(0x55b249de41b0 msgr2=0x55b249e975e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.677+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1352072956 shutdown_connections 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.678+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1352072956 wait complete. 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.678+0000 7fdf5199fb80 1 -- start start 2024-05-08T22:00:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.679+0000 7fdf5199fb80 1 Processor -- start 2024-05-08T22:00:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 0x55b249e9b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 0x55b249e9b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249e9ece0 con 0x55b249e85a80 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249e9ee50 con 0x55b249e863e0 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b249e9f010 con 0x55b249e872d0 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf5199fb80 10 client.0 _ll_register_callbacks cb 0x55b249d7bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 0x55b249e9b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 0x55b249e9b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 0x55b249e9b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:42562/0 (socket says 172.21.15.64:42562) 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 0x55b249e9b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51702/0 (socket says 172.21.15.64:51702) 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1608729579 learned_addr learned my addr 172.21.15.64:0/1608729579 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3f7fe700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1608729579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 msgr2=0x55b249e9e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1608729579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 msgr2=0x55b249e9b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 0x55b249e9b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.680+0000 7fdf3e7fc700 1 -- 172.21.15.64:0/1608729579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf40109c90 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf3effd700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 0x55b249e9b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf3e7fc700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 0x55b249e9b870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf34005b40 tx=0x7fdf34005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55cc7221eb81c26f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf22ff5700 1 -- 172.21.15.64:0/1608729579 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400b9f0 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf22ff5700 1 -- 172.21.15.64:0/1608729579 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf3400bb50 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf3009dda0 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.681+0000 7fdf22ff5700 1 -- 172.21.15.64:0/1608729579 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf34026910 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 10 client.25421 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b249c5bc70 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf22ff5700 1 -- 172.21.15.64:0/1608729579 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3009dda0 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf22ff5700 1 -- 172.21.15.64:0/1608729579 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf3400f460 con 0x55b249e863e0 2024-05-08T22:00:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 10 client.25421 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 20 client.25421 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249f02cc0 0x55b249f050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 4 client.25421 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b249dcee60 con 0x55b249f02cc0 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.682+0000 7fdf3effd700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249f02cc0 0x55b249f050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.683+0000 7fdf22ff5700 10 client.25421 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.683+0000 7fdf3effd700 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249f02cc0 0x55b249f050b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb300791cfc50577 server_cookie=7e9282468f05c79a in_seq=0 out_seq=0 2024-05-08T22:00:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.699+0000 7fdf22ff5700 1 -- 172.21.15.64:0/1608729579 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b249dcee60 con 0x55b249f02cc0 2024-05-08T22:00:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.699+0000 7fdf22ff5700 10 client.25421 handle_command_reply: tid=0 2024-05-08T22:00:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.699+0000 7fdf5199fb80 1 client.25421 shutdown 2024-05-08T22:00:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 msgr2=0x55b249e9b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 0x55b249e9b870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf34005b40 tx=0x7fdf34005710 comp rx=0 tx=0).stop 2024-05-08T22:00:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 shutdown_connections 2024-05-08T22:00:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b249f02cc0 0x55b249f050b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf22ff5700 0 client.25421 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b249e872d0 0x55b249e9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b249e85a80 0x55b249e9b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 --2- 172.21.15.64:0/1608729579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b249e863e0 0x55b249e9b870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 >> 172.21.15.64:0/1608729579 conn(0x55b249de41b0 msgr2=0x55b249e82020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 shutdown_connections 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.701+0000 7fdf5199fb80 1 -- 172.21.15.64:0/1608729579 wait complete. 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf5199fb80 20 client.25421 trim_cache size 0 max 16384 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:00:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fdf280a11a0 msgr2=0x7fdf280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fdf280a11a0 0x7fdf280a3660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf38003ee0 tx=0x7fdf38004320 comp rx=0 tx=0).stop 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 msgr2=0x7fdf4006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf4006f860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000e9a0 tx=0x7fdf3000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 shutdown_connections 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fdf280a11a0 0x7fdf280a3660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf40102a40 0x7fdf401a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100f30 0x7fdf4006f860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 --2- 172.21.15.64:0/801631148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40102080 0x7fdf4006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 >> 172.21.15.64:0/801631148 conn(0x7fdf400fc4a0 msgr2=0x7fdf401054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 shutdown_connections 2024-05-08T22:00:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:09.702+0000 7fdf457e5700 1 -- 172.21.15.64:0/801631148 wait complete. 2024-05-08T22:00:09.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:09 smithi160 ceph-mon[46962]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 1.79k op/s 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:09 smithi186 ceph-mon[46619]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 1.79k op/s 2024-05-08T22:00:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:09 smithi064 ceph-mon[42500]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 1.79k op/s 2024-05-08T22:00:11.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 320.0 s, 2689.4 tps, lat 36.943 ms stddev 43.769 2024-05-08T22:00:11.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:11 smithi160 ceph-mon[46962]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:11 smithi186 ceph-mon[46619]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-05-08T22:00:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:11 smithi064 ceph-mon[42500]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-05-08T22:00:13.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:13 smithi160 ceph-mon[46962]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 729 op/s 2024-05-08T22:00:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:13 smithi186 ceph-mon[46619]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 729 op/s 2024-05-08T22:00:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:13 smithi064 ceph-mon[42500]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 729 op/s 2024-05-08T22:00:15.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:15 smithi160 ceph-mon[46962]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 770 op/s 2024-05-08T22:00:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:15 smithi186 ceph-mon[46619]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 770 op/s 2024-05-08T22:00:15.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:15 smithi064 ceph-mon[42500]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 770 op/s 2024-05-08T22:00:16.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 325.0 s, 2966.2 tps, lat 33.546 ms stddev 40.609 2024-05-08T22:00:17.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:17 smithi160 ceph-mon[46962]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 677 op/s 2024-05-08T22:00:17.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:17 smithi186 ceph-mon[46619]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 677 op/s 2024-05-08T22:00:17.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:17 smithi064 ceph-mon[42500]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 677 op/s 2024-05-08T22:00:19.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:19 smithi160 ceph-mon[46962]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 704 op/s 2024-05-08T22:00:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:19 smithi186 ceph-mon[46619]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 704 op/s 2024-05-08T22:00:19.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:19 smithi064 ceph-mon[42500]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 704 op/s 2024-05-08T22:00:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 330.0 s, 2189.8 tps, lat 45.996 ms stddev 53.295 2024-05-08T22:00:21.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:21 smithi160 ceph-mon[46962]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 755 op/s 2024-05-08T22:00:21.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:21 smithi186 ceph-mon[46619]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 755 op/s 2024-05-08T22:00:21.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:21 smithi064 ceph-mon[42500]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 755 op/s 2024-05-08T22:00:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 -- 172.21.15.64:0/1001123825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102090 msgr2=0x7f257c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 --2- 172.21.15.64:0/1001123825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102090 0x7f257c102510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f256c009990 tx=0x7f256c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 -- 172.21.15.64:0/1001123825 shutdown_connections 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 --2- 172.21.15.64:0/1001123825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 --2- 172.21.15.64:0/1001123825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102a50 0x7f257c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 --2- 172.21.15.64:0/1001123825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102090 0x7f257c102510 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 -- 172.21.15.64:0/1001123825 >> 172.21.15.64:0/1001123825 conn(0x7f257c0fc470 msgr2=0x7f257c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 -- 172.21.15.64:0/1001123825 shutdown_connections 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.864+0000 7f258322a700 1 -- 172.21.15.64:0/1001123825 wait complete. 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 Processor -- start 2024-05-08T22:00:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 -- start start 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102090 0x7f257c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102a50 0x7f257c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0731d0 con 0x7f257c102a50 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c073340 con 0x7f257c100eb0 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f258322a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c073500 con 0x7f257c102090 2024-05-08T22:00:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60606/0 (socket says 172.21.15.64:60606) 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f2580fc6700 1 -- 172.21.15.64:0/3844416294 learned_addr learned my addr 172.21.15.64:0/3844416294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f257bfff700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102090 0x7f257c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.865+0000 7f25817c7700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102a50 0x7f257c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2580fc6700 1 -- 172.21.15.64:0/3844416294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102090 msgr2=0x7f257c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2580fc6700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102090 0x7f257c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2580fc6700 1 -- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102a50 msgr2=0x7f257c19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2580fc6700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102a50 0x7f257c19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2580fc6700 1 -- 172.21.15.64:0/3844416294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0095f0 con 0x7f257c100eb0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f257bfff700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102090 0x7f257c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2580fc6700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c196f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f257000b550 tx=0x7f257000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2579ffb700 1 -- 172.21.15.64:0/3844416294 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570011830 con 0x7f257c100eb0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c073780 con 0x7f257c100eb0 2024-05-08T22:00:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2579ffb700 1 -- 172.21.15.64:0/3844416294 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2570011e50 con 0x7f257c100eb0 2024-05-08T22:00:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f2579ffb700 1 -- 172.21.15.64:0/3844416294 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570010de0 con 0x7f257c100eb0 2024-05-08T22:00:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.866+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c073d60 con 0x7f257c100eb0 2024-05-08T22:00:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.869+0000 7f2579ffb700 1 -- 172.21.15.64:0/3844416294 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2570010480 con 0x7f257c100eb0 2024-05-08T22:00:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.869+0000 7f2579ffb700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25640a1120 0x7f25640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.870+0000 7f2579ffb700 1 -- 172.21.15.64:0/3844416294 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25700cc320 con 0x7f257c100eb0 2024-05-08T22:00:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.870+0000 7f257bfff700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25640a1120 0x7f25640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.870+0000 7f258f3e4b80 1 Processor -- start 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.870+0000 7f257bfff700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25640a1120 0x7f25640a35e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f256c00b5c0 tx=0x7f256c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.870+0000 7f258f3e4b80 1 -- start start 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 0x5560b83dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b83330f0 con 0x5560b83dc3e0 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b8330ab0 con 0x5560b83dba80 2024-05-08T22:00:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b832c3a0 con 0x5560b83dd2d0 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 0x5560b83dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 0x5560b83dbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60616/0 (socket says 172.21.15.64:60616) 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 -- 172.21.15.64:0/1824222538 learned_addr learned my addr 172.21.15.64:0/1824222538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 -- 172.21.15.64:0/1824222538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 msgr2=0x5560b83dd750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83dd750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 -- 172.21.15.64:0/1824222538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 msgr2=0x5560b83dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.871+0000 7f2580fc6700 1 -- 172.21.15.64:0/1824222538 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560b81ad350 con 0x5560b83dba80 2024-05-08T22:00:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.872+0000 7f2580fc6700 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 0x5560b83dbea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25700996b0 tx=0x7f25700994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6afceac8d9426f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.872+0000 7f2562ffd700 1 -- 172.21.15.64:0/1824222538 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570099080 con 0x5560b83dba80 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.872+0000 7f2562ffd700 1 -- 172.21.15.64:0/1824222538 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2570099e20 con 0x5560b83dba80 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.872+0000 7f2562ffd700 1 -- 172.21.15.64:0/1824222538 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257009d120 con 0x5560b83dba80 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 -- 172.21.15.64:0/1824222538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 msgr2=0x5560b83dbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 0x5560b83dbea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25700996b0 tx=0x7f25700994f0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 -- 172.21.15.64:0/1824222538 shutdown_connections 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dba80 0x5560b83dbea0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83dd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/1824222538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.873+0000 7f258f3e4b80 1 -- 172.21.15.64:0/1824222538 >> 172.21.15.64:0/1824222538 conn(0x5560b833a1b0 msgr2=0x5560b83d8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.876+0000 7f258f3e4b80 1 -- 172.21.15.64:0/1824222538 shutdown_connections 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 -- 172.21.15.64:0/1824222538 wait complete. 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 -- start start 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 Processor -- start 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 0x5560b83f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83fde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b82c8500 con 0x5560b83dba80 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b82c81a0 con 0x5560b83dc3e0 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.878+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b81af640 con 0x5560b83dd2d0 2024-05-08T22:00:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f258f3e4b80 10 client.0 _ll_register_callbacks cb 0x5560b82d1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 0x5560b83f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 0x5560b83f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56590/0 (socket says 172.21.15.64:56590) 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f2580fc6700 1 -- 172.21.15.64:0/2822041081 learned_addr learned my addr 172.21.15.64:0/2822041081 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83fde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f257bfff700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 -- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 msgr2=0x5560b83f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 -- 172.21.15.64:0/2822041081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 msgr2=0x5560b83f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 0x5560b83f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 -- 172.21.15.64:0/2822041081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c073d60 con 0x5560b83dd2d0 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f2580fc6700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 0x5560b83f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f25817c7700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83fde00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f257400a7c0 tx=0x7f257400aad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8352bf5a4dd8b043 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.879+0000 7f255bfff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:00:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f255bfff700 1 -- 172.21.15.64:0/2822041081 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574010040 con 0x5560b83dd2d0 2024-05-08T22:00:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560b8285210 con 0x5560b83dd2d0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f255bfff700 1 -- 172.21.15.64:0/2822041081 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2574013790 con 0x5560b83dd2d0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f255bfff700 1 -- 172.21.15.64:0/2822041081 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25740091d0 con 0x5560b83dd2d0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 10 client.25333 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5560b8159b00 con 0x5560b83dd2d0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f255bfff700 1 -- 172.21.15.64:0/2822041081 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2574023020 con 0x5560b83dd2d0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f255bfff700 1 -- 172.21.15.64:0/2822041081 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f257401fc30 con 0x5560b83dd2d0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 10 client.25333 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 20 client.25333 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b83e85f0 0x5560b83eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 4 client.25333 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560b83ab7b0 con 0x5560b83e85f0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.880+0000 7f2580fc6700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b83e85f0 0x5560b83eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.881+0000 7f255bfff700 10 client.25333 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.881+0000 7f2580fc6700 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b83e85f0 0x5560b83eaa10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbac80f205ed9b7 server_cookie=cbe7d46092b4270a in_seq=0 out_seq=0 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.881+0000 7f255bfff700 1 -- 172.21.15.64:0/2822041081 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560b83ab7b0 con 0x5560b83e85f0 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.881+0000 7f255bfff700 10 client.25333 handle_command_reply: tid=0 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.881+0000 7f258f3e4b80 1 client.25333 shutdown 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 msgr2=0x5560b83fde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83fde00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f257400a7c0 tx=0x7f257400aad0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 shutdown_connections 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b83e85f0 0x5560b83eaa10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dd2d0 0x5560b83fde00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dba80 0x5560b83f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2822041081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 >> 172.21.15.64:0/2822041081 conn(0x5560b833a1b0 msgr2=0x5560b83d6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f255bfff700 0 client.25333 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 shutdown_connections 2024-05-08T22:00:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.882+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2822041081 wait complete. 2024-05-08T22:00:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.883+0000 7f258f3e4b80 20 client.25333 trim_cache size 0 max 16384 2024-05-08T22:00:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.908+0000 7f258f3e4b80 1 Processor -- start 2024-05-08T22:00:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.908+0000 7f258f3e4b80 1 -- start start 2024-05-08T22:00:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.909+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83e4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.909+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.909+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 0x5560b83fddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.909+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b83d1330 con 0x5560b83dc3e0 2024-05-08T22:00:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.909+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b83e5a20 con 0x5560b83dd2d0 2024-05-08T22:00:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.909+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b83db5e0 con 0x5560b83dba80 2024-05-08T22:00:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.914+0000 7f25817c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 0x5560b83fddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.921+0000 7f25817c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 0x5560b83fddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60622/0 (socket says 172.21.15.64:60622) 2024-05-08T22:00:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.921+0000 7f25817c7700 1 -- 172.21.15.64:0/2029299728 learned_addr learned my addr 172.21.15.64:0/2029299728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.925+0000 7f257bfff700 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.926+0000 7f25817c7700 1 -- 172.21.15.64:0/2029299728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 msgr2=0x5560b83e4520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:00:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.926+0000 7f25817c7700 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83e4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.926+0000 7f25817c7700 1 -- 172.21.15.64:0/2029299728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 msgr2=0x5560b83e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.926+0000 7f25817c7700 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.926+0000 7f25817c7700 1 -- 172.21.15.64:0/2029299728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560b82f2500 con 0x5560b83dd2d0 2024-05-08T22:00:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.936+0000 7f25817c7700 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 0x5560b83fddc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2574000f80 tx=0x7f25740092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628d423cc5bbc852 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.948+0000 7f25637fe700 1 -- 172.21.15.64:0/2029299728 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574014690 con 0x5560b83dd2d0 2024-05-08T22:00:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.948+0000 7f25637fe700 1 -- 172.21.15.64:0/2029299728 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2574023030 con 0x5560b83dd2d0 2024-05-08T22:00:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.948+0000 7f25637fe700 1 -- 172.21.15.64:0/2029299728 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257401f690 con 0x5560b83dd2d0 2024-05-08T22:00:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.953+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2029299728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 msgr2=0x5560b83fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.954+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 0x5560b83fddc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2574000f80 tx=0x7f25740092c0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.954+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2029299728 shutdown_connections 2024-05-08T22:00:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.954+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dd2d0 0x5560b83fddc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.954+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.954+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/2029299728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dc3e0 0x5560b83e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.954+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2029299728 >> 172.21.15.64:0/2029299728 conn(0x5560b833a1b0 msgr2=0x5560b83e38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.964+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2029299728 shutdown_connections 2024-05-08T22:00:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.964+0000 7f258f3e4b80 1 -- 172.21.15.64:0/2029299728 wait complete. 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.964+0000 7f258f3e4b80 1 -- start start 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.964+0000 7f258f3e4b80 1 Processor -- start 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dd2d0 0x5560b84677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b8467d10 con 0x5560b83dd2d0 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b8467e50 con 0x5560b83dc3e0 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b8467fe0 con 0x5560b83dba80 2024-05-08T22:00:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f258f3e4b80 10 client.0 _ll_register_callbacks cb 0x5560b82d1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83fdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60624/0 (socket says 172.21.15.64:60624) 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f2580fc6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 -- 172.21.15.64:0/3160351739 learned_addr learned my addr 172.21.15.64:0/3160351739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 -- 172.21.15.64:0/3160351739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 msgr2=0x5560b83d7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83d7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 -- 172.21.15.64:0/3160351739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dd2d0 msgr2=0x5560b84677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dd2d0 0x5560b84677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.965+0000 7f257bfff700 1 -- 172.21.15.64:0/3160351739 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560b8159b00 con 0x5560b83dc3e0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.967+0000 7f257bfff700 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83fdd20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f256c009990 tx=0x7f256c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5aa4334039dc027 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.967+0000 7f255bfff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.967+0000 7f255bfff700 1 -- 172.21.15.64:0/3160351739 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c023c50 con 0x5560b83dc3e0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.967+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2570099e20 con 0x5560b83dc3e0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.967+0000 7f255bfff700 1 -- 172.21.15.64:0/3160351739 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f256c023db0 con 0x5560b83dc3e0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f255bfff700 1 -- 172.21.15.64:0/3160351739 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f256c030410 con 0x5560b83dc3e0 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 10 client.25436 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5560b81b1c70 con 0x5560b83dc3e0 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f255bfff700 1 -- 172.21.15.64:0/3160351739 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2570099e20 con 0x5560b83dc3e0 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f255bfff700 1 -- 172.21.15.64:0/3160351739 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f256c0105c0 con 0x5560b83dc3e0 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 10 client.25436 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 10 client.25436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 20 client.25436 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b8469aa0 0x5560b846bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 4 client.25436 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560b8324e60 con 0x5560b8469aa0 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.968+0000 7f2580fc6700 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b8469aa0 0x5560b846bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.969+0000 7f2580fc6700 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b8469aa0 0x5560b846bec0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb042a6bb11dd27 server_cookie=8e9e45f1378ce87f in_seq=0 out_seq=0 2024-05-08T22:00:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.969+0000 7f255bfff700 10 client.25436 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f255bfff700 1 -- 172.21.15.64:0/3160351739 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560b8324e60 con 0x5560b8469aa0 2024-05-08T22:00:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f255bfff700 10 client.25436 handle_command_reply: tid=0 2024-05-08T22:00:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 client.25436 shutdown 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 msgr2=0x5560b83fdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83fdd20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f256c009990 tx=0x7f256c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 shutdown_connections 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5560b8469aa0 0x5560b846bec0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560b83dd2d0 0x5560b84677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560b83dba80 0x5560b83d7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 --2- 172.21.15.64:0/3160351739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560b83dc3e0 0x5560b83fdd20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 >> 172.21.15.64:0/3160351739 conn(0x5560b833a1b0 msgr2=0x5560b83ef370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.989+0000 7f255bfff700 0 client.25436 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.990+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 shutdown_connections 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.990+0000 7f258f3e4b80 1 -- 172.21.15.64:0/3160351739 wait complete. 2024-05-08T22:00:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.990+0000 7f258f3e4b80 20 client.25436 trim_cache size 0 max 16384 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.990+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25640a1120 msgr2=0x7f25640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.990+0000 7f258322a700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25640a1120 0x7f25640a35e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f256c00b5c0 tx=0x7f256c003860 comp rx=0 tx=0).stop 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.991+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 msgr2=0x7f257c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.991+0000 7f258322a700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c196f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f257000b550 tx=0x7f257000b860 comp rx=0 tx=0).stop 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 shutdown_connections 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25640a1120 0x7f25640a35e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f257c100eb0 0x7f257c196f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f257c102a50 0x7f257c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 --2- 172.21.15.64:0/3844416294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f257c102090 0x7f257c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 >> 172.21.15.64:0/3844416294 conn(0x7f257c0fc470 msgr2=0x7f257c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 shutdown_connections 2024-05-08T22:00:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:22.992+0000 7f258322a700 1 -- 172.21.15.64:0/3844416294 wait complete. 2024-05-08T22:00:23.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:23 smithi160 ceph-mon[46962]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 719 op/s 2024-05-08T22:00:23.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:23 smithi186 ceph-mon[46619]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 719 op/s 2024-05-08T22:00:23.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:23 smithi064 ceph-mon[42500]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 719 op/s 2024-05-08T22:00:25.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:25 smithi186 ceph-mon[46619]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 788 op/s 2024-05-08T22:00:25.971 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:25 smithi064 ceph-mon[42500]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 788 op/s 2024-05-08T22:00:26.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:25 smithi160 ceph-mon[46962]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 788 op/s 2024-05-08T22:00:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 335.0 s, 2221.6 tps, lat 44.986 ms stddev 51.276 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:27 smithi186 ceph-mon[46619]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 710 op/s 2024-05-08T22:00:27.974 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:27 smithi064 ceph-mon[42500]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 710 op/s 2024-05-08T22:00:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:27 smithi160 ceph-mon[46962]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 710 op/s 2024-05-08T22:00:29.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:29 smithi186 ceph-mon[46619]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-05-08T22:00:29.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:29 smithi064 ceph-mon[42500]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-05-08T22:00:30.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:29 smithi160 ceph-mon[46962]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-05-08T22:00:30.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:30 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:31.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:30 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:31.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:30 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 340.0 s, 1613.0 tps, lat 62.185 ms stddev 71.293 2024-05-08T22:00:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:31 smithi064 ceph-mon[42500]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 862 op/s 2024-05-08T22:00:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:31 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:00:32.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:31 smithi160 ceph-mon[46962]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 862 op/s 2024-05-08T22:00:32.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:32.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:31 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:00:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:31 smithi186 ceph-mon[46619]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 862 op/s 2024-05-08T22:00:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:31 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:00:33.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:33 smithi064 ceph-mon[42500]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 849 op/s 2024-05-08T22:00:34.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:33 smithi160 ceph-mon[46962]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 849 op/s 2024-05-08T22:00:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:33 smithi186 ceph-mon[46619]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 849 op/s 2024-05-08T22:00:35.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:35 smithi064 ceph-mon[42500]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 916 op/s 2024-05-08T22:00:36.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:35 smithi160 ceph-mon[46962]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 916 op/s 2024-05-08T22:00:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:35 smithi186 ceph-mon[46619]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 916 op/s 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 -- 172.21.15.64:0/1601261913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678101990 msgr2=0x7f1678108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 --2- 172.21.15.64:0/1601261913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678101990 0x7f1678108240 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f167000b0d0 tx=0x7f167000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 -- 172.21.15.64:0/1601261913 shutdown_connections 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 --2- 172.21.15.64:0/1601261913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678101990 0x7f1678108240 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 --2- 172.21.15.64:0/1601261913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 --2- 172.21.15.64:0/1601261913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1678100ff0 0x7f1678101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.146+0000 7f167ee89700 1 -- 172.21.15.64:0/1601261913 >> 172.21.15.64:0/1601261913 conn(0x7f16780fb390 msgr2=0x7f16780fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.148+0000 7f167ee89700 1 -- 172.21.15.64:0/1601261913 shutdown_connections 2024-05-08T22:00:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.148+0000 7f167ee89700 1 -- 172.21.15.64:0/1601261913 wait complete. 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 Processor -- start 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 -- start start 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678103d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678100ff0 0x7f16781023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1678101990 0x7f1678102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781030b0 con 0x7f1678100ff0 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678103220 con 0x7f1678101990 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167ee89700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781a0fe0 con 0x7f16780ffe00 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f1677fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678100ff0 0x7f16781023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678103d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678103d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:35872/0 (socket says 172.21.15.64:35872) 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f1677fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678100ff0 0x7f16781023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37298/0 (socket says 172.21.15.64:37298) 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 -- 172.21.15.64:0/4108079653 learned_addr learned my addr 172.21.15.64:0/4108079653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167d426700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1678101990 0x7f1678102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 -- 172.21.15.64:0/4108079653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1678101990 msgr2=0x7f1678102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1678101990 0x7f1678102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 -- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678100ff0 msgr2=0x7f16781023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678100ff0 0x7f16781023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.149+0000 7f167cc25700 1 -- 172.21.15.64:0/4108079653 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1670009d00 con 0x7f16780ffe00 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.150+0000 7f167cc25700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678103d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f166c00e970 tx=0x7f166c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.150+0000 7f1675ffb700 1 -- 172.21.15.64:0/4108079653 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c009790 con 0x7f16780ffe00 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.150+0000 7f1675ffb700 1 -- 172.21.15.64:0/4108079653 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f166c00c7e0 con 0x7f16780ffe00 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.150+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16781a1210 con 0x7f16780ffe00 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.150+0000 7f1675ffb700 1 -- 172.21.15.64:0/4108079653 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c003c70 con 0x7f16780ffe00 2024-05-08T22:00:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.150+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16781a1770 con 0x7f16780ffe00 2024-05-08T22:00:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.152+0000 7f1675ffb700 1 -- 172.21.15.64:0/4108079653 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f166c00cc50 con 0x7f16780ffe00 2024-05-08T22:00:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.153+0000 7f1675ffb700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16680a11a0 0x7f16680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.153+0000 7f1677fff700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16680a11a0 0x7f16680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.153+0000 7f1675ffb700 1 -- 172.21.15.64:0/4108079653 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f166c014070 con 0x7f16780ffe00 2024-05-08T22:00:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.153+0000 7f1677fff700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16680a11a0 0x7f16680a3660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1664009c60 tx=0x7f1664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 Processor -- start 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 -- start start 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2af9a80 0x55daa2af9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2afacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 0x55daa2afb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2a510f0 con 0x55daa2afb2d0 2024-05-08T22:00:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2a4eab0 con 0x55daa2afa3e0 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f168b043b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2a4a3a0 con 0x55daa2af9a80 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f167d426700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 0x55daa2afb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f167d426700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 0x55daa2afb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37312/0 (socket says 172.21.15.64:37312) 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f167d426700 1 -- 172.21.15.64:0/1672769119 learned_addr learned my addr 172.21.15.64:0/1672769119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f167cc25700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2af9a80 0x55daa2af9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.154+0000 7f1677fff700 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2afacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167d426700 1 -- 172.21.15.64:0/1672769119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2af9a80 msgr2=0x55daa2af9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167d426700 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2af9a80 0x55daa2af9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167d426700 1 -- 172.21.15.64:0/1672769119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 msgr2=0x55daa2afacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167d426700 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2afacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167d426700 1 -- 172.21.15.64:0/1672769119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daa28cb350 con 0x55daa2afb2d0 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f1677fff700 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2afacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167cc25700 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2af9a80 0x55daa2af9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f167d426700 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 0x55daa2afb750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1670015000 tx=0x7f1670009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6e3ac455efe817 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f1662ffd700 1 -- 172.21.15.64:0/1672769119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000e040 con 0x55daa2afb2d0 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f1662ffd700 1 -- 172.21.15.64:0/1672769119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f167000baa0 con 0x55daa2afb2d0 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.155+0000 7f1662ffd700 1 -- 172.21.15.64:0/1672769119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670005070 con 0x55daa2afb2d0 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 -- 172.21.15.64:0/1672769119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 msgr2=0x55daa2afb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 0x55daa2afb750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1670015000 tx=0x7f1670009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 -- 172.21.15.64:0/1672769119 shutdown_connections 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afb2d0 0x55daa2afb750 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2af9a80 0x55daa2af9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 --2- 172.21.15.64:0/1672769119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2afacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 -- 172.21.15.64:0/1672769119 >> 172.21.15.64:0/1672769119 conn(0x55daa2a581b0 msgr2=0x55daa2af6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 -- 172.21.15.64:0/1672769119 shutdown_connections 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.156+0000 7f168b043b80 1 -- 172.21.15.64:0/1672769119 wait complete. 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 -- start start 2024-05-08T22:00:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 Processor -- start 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2af9a80 0x55daa2b0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afa3e0 0x55daa2b0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b13590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa29e6500 con 0x55daa2afa3e0 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa29e61a0 con 0x55daa2af9a80 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa28cd640 con 0x55daa2afb2d0 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f168b043b80 10 client.0 _ll_register_callbacks cb 0x55daa29efad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.157+0000 7f167d426700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b13590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:35890/0 (socket says 172.21.15.64:35890) 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 -- 172.21.15.64:0/787073351 learned_addr learned my addr 172.21.15.64:0/787073351 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 -- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2af9a80 msgr2=0x55daa2b0c010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2af9a80 0x55daa2b0c010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 -- 172.21.15.64:0/787073351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afa3e0 msgr2=0x55daa2b0c550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afa3e0 0x55daa2b0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 -- 172.21.15.64:0/787073351 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1664017070 con 0x55daa2afb2d0 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.158+0000 7f167d426700 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b13590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1670007b60 tx=0x7f1670009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec940caa2576c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.159+0000 7f1657fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.159+0000 7f1657fff700 1 -- 172.21.15.64:0/787073351 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000e040 con 0x55daa2afb2d0 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.159+0000 7f1657fff700 1 -- 172.21.15.64:0/787073351 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16700054e0 con 0x55daa2afb2d0 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.159+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daa29a3210 con 0x55daa2afb2d0 2024-05-08T22:00:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.162+0000 7f1657fff700 1 -- 172.21.15.64:0/787073351 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16700056a0 con 0x55daa2afb2d0 2024-05-08T22:00:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.162+0000 7f168b043b80 10 client.25348 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.162+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f1678108b90 con 0x55daa2afb2d0 2024-05-08T22:00:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.162+0000 7f1657fff700 1 -- 172.21.15.64:0/787073351 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55daa29a3210 con 0x55daa2afb2d0 2024-05-08T22:00:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.162+0000 7f1657fff700 1 -- 172.21.15.64:0/787073351 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f167000b100 con 0x55daa2afb2d0 2024-05-08T22:00:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.162+0000 7f168b043b80 10 client.25348 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.163+0000 7f168b043b80 10 client.25348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.163+0000 7f168b043b80 20 client.25348 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.163+0000 7f168b043b80 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b07250 0x55daa2b09670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.163+0000 7f168b043b80 4 client.25348 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:00:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.163+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daa2ac97b0 con 0x55daa2b07250 2024-05-08T22:00:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.163+0000 7f167cc25700 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b07250 0x55daa2b09670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.164+0000 7f167cc25700 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b07250 0x55daa2b09670 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9ba9dc20ed6df server_cookie=f22947da1fb71d8c in_seq=0 out_seq=0 2024-05-08T22:00:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.164+0000 7f1657fff700 10 client.25348 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.164+0000 7f1657fff700 1 -- 172.21.15.64:0/787073351 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55daa2ac97b0 con 0x55daa2b07250 2024-05-08T22:00:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.164+0000 7f1657fff700 10 client.25348 handle_command_reply: tid=0 2024-05-08T22:00:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.164+0000 7f168b043b80 1 client.25348 shutdown 2024-05-08T22:00:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 msgr2=0x55daa2b13590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b13590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1670007b60 tx=0x7f1670009550 comp rx=0 tx=0).stop 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 shutdown_connections 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2af9a80 0x55daa2b0c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b07250 0x55daa2b09670 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b13590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f1657fff700 0 client.25348 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 --2- 172.21.15.64:0/787073351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2afa3e0 0x55daa2b0c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 >> 172.21.15.64:0/787073351 conn(0x55daa2a581b0 msgr2=0x55daa2af4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.165+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 shutdown_connections 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.166+0000 7f168b043b80 1 -- 172.21.15.64:0/787073351 wait complete. 2024-05-08T22:00:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.166+0000 7f168b043b80 20 client.25348 trim_cache size 0 max 16384 2024-05-08T22:00:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 Processor -- start 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 -- start start 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2af9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afa3e0 0x55daa2b13550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afb2d0 0x55daa2afb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2af19e0 con 0x55daa2af9a80 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2b040a0 con 0x55daa2afb2d0 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.188+0000 7f168b043b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2b070c0 con 0x55daa2afa3e0 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167cc25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2af9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167cc25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2af9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37340/0 (socket says 172.21.15.64:37340) 2024-05-08T22:00:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167cc25700 1 -- 172.21.15.64:0/2519066145 learned_addr learned my addr 172.21.15.64:0/2519066145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afb2d0 0x55daa2afb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f1677fff700 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afa3e0 0x55daa2b13550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 -- 172.21.15.64:0/2519066145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afa3e0 msgr2=0x55daa2b13550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afa3e0 0x55daa2b13550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 -- 172.21.15.64:0/2519066145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 msgr2=0x55daa2af9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2af9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 -- 172.21.15.64:0/2519066145 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daa2877b00 con 0x55daa2afb2d0 2024-05-08T22:00:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167cc25700 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2af9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.189+0000 7f167d426700 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afb2d0 0x55daa2afb6b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16700097f0 tx=0x7f1670000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b256d015fdb3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.190+0000 7f16637fe700 1 -- 172.21.15.64:0/2519066145 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167002e4c0 con 0x55daa2afb2d0 2024-05-08T22:00:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.190+0000 7f16637fe700 1 -- 172.21.15.64:0/2519066145 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1670034070 con 0x55daa2afb2d0 2024-05-08T22:00:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.190+0000 7f16637fe700 1 -- 172.21.15.64:0/2519066145 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167002e960 con 0x55daa2afb2d0 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 -- 172.21.15.64:0/2519066145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afb2d0 msgr2=0x55daa2afb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afb2d0 0x55daa2afb6b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16700097f0 tx=0x7f1670000f40 comp rx=0 tx=0).stop 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 -- 172.21.15.64:0/2519066145 shutdown_connections 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afb2d0 0x55daa2afb6b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2af9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 --2- 172.21.15.64:0/2519066145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afa3e0 0x55daa2b13550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 -- 172.21.15.64:0/2519066145 >> 172.21.15.64:0/2519066145 conn(0x55daa2a581b0 msgr2=0x55daa2b0c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 -- 172.21.15.64:0/2519066145 shutdown_connections 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 -- 172.21.15.64:0/2519066145 wait complete. 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 -- start start 2024-05-08T22:00:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.191+0000 7f168b043b80 1 Processor -- start 2024-05-08T22:00:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.193+0000 7f168b043b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2b0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.193+0000 7f168b043b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2b0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.193+0000 7f168b043b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.193+0000 7f168b043b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2b12680 con 0x55daa2af9a80 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.193+0000 7f168b043b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2b127f0 con 0x55daa2afa3e0 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.193+0000 7f168b043b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2b129b0 con 0x55daa2afb2d0 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f168b043b80 10 client.0 _ll_register_callbacks cb 0x55daa29efad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f1677fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2b0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f1677fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2b0f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:55772/0 (socket says 172.21.15.64:55772) 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f1677fff700 1 -- 172.21.15.64:0/3159875246 learned_addr learned my addr 172.21.15.64:0/3159875246 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167cc25700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2b0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167d426700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167cc25700 1 -- 172.21.15.64:0/3159875246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 msgr2=0x55daa2b120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167cc25700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167cc25700 1 -- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 msgr2=0x55daa2b0f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167cc25700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2b0f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.194+0000 7f167cc25700 1 -- 172.21.15.64:0/3159875246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16781a1210 con 0x55daa2af9a80 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.195+0000 7f167cc25700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2b0ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f166c0963d0 tx=0x7f166c003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d29e79e1fb47c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.195+0000 7f1657fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.195+0000 7f1657fff700 1 -- 172.21.15.64:0/3159875246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0a2350 con 0x55daa2af9a80 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.195+0000 7f1657fff700 1 -- 172.21.15.64:0/3159875246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f166c0ca690 con 0x55daa2af9a80 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.195+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f167000baa0 con 0x55daa2af9a80 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f1657fff700 1 -- 172.21.15.64:0/3159875246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f166c098270 con 0x55daa2af9a80 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 10 client.15708 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f1678108b90 con 0x55daa2af9a80 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f1657fff700 1 -- 172.21.15.64:0/3159875246 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f167000baa0 con 0x55daa2af9a80 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f1657fff700 1 -- 172.21.15.64:0/3159875246 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f166c09a130 con 0x55daa2af9a80 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 20 client.15708 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b76c00 0x55daa2b79020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 4 client.15708 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55daa2a42e60 con 0x55daa2b76c00 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.196+0000 7f1677fff700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b76c00 0x55daa2b79020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.197+0000 7f1657fff700 10 client.15708 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.197+0000 7f1677fff700 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b76c00 0x55daa2b79020 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8231f26c317fa84f server_cookie=5c22c1266ae3edbc in_seq=0 out_seq=0 2024-05-08T22:00:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f1657fff700 1 -- 172.21.15.64:0/3159875246 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55daa2a42e60 con 0x55daa2b76c00 2024-05-08T22:00:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f1657fff700 10 client.15708 handle_command_reply: tid=0 2024-05-08T22:00:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 client.15708 shutdown 2024-05-08T22:00:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 msgr2=0x55daa2b0ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2b0ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f166c0963d0 tx=0x7f166c003710 comp rx=0 tx=0).stop 2024-05-08T22:00:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:00:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:00:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:00:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:00:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 shutdown_connections 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55daa2b76c00 0x55daa2b79020 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55daa2afb2d0 0x55daa2b120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daa2af9a80 0x55daa2b0ede0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 --2- 172.21.15.64:0/3159875246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55daa2afa3e0 0x55daa2b0f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 >> 172.21.15.64:0/3159875246 conn(0x55daa2a581b0 msgr2=0x55daa2af6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.209+0000 7f1657fff700 0 client.15708 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.210+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 shutdown_connections 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.210+0000 7f168b043b80 1 -- 172.21.15.64:0/3159875246 wait complete. 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.210+0000 7f168b043b80 20 client.15708 trim_cache size 0 max 16384 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16680a11a0 msgr2=0x7f16680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16680a11a0 0x7f16680a3660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1664009c60 tx=0x7f1664009260 comp rx=0 tx=0).stop 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 msgr2=0x7f1678103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678103d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f166c00e970 tx=0x7f166c00ec80 comp rx=0 tx=0).stop 2024-05-08T22:00:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 shutdown_connections 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f16680a11a0 0x7f16680a3660 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1678101990 0x7f1678102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16780ffe00 0x7f1678103d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 --2- 172.21.15.64:0/4108079653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1678100ff0 0x7f16781023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 >> 172.21.15.64:0/4108079653 conn(0x7f16780fb390 msgr2=0x7f16780fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 shutdown_connections 2024-05-08T22:00:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:36.211+0000 7f167ee89700 1 -- 172.21.15.64:0/4108079653 wait complete. 2024-05-08T22:00:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 345.0 s, 2389.0 tps, lat 41.708 ms stddev 46.711 2024-05-08T22:00:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:37 smithi160 ceph-mon[46962]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 874 op/s 2024-05-08T22:00:38.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:37 smithi186 ceph-mon[46619]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 874 op/s 2024-05-08T22:00:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:37 smithi064 ceph-mon[42500]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 874 op/s 2024-05-08T22:00:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:39 smithi160 ceph-mon[46962]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 19 MiB/s wr, 914 op/s 2024-05-08T22:00:40.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:39 smithi186 ceph-mon[46619]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 19 MiB/s wr, 914 op/s 2024-05-08T22:00:40.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:39 smithi064 ceph-mon[42500]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 19 MiB/s wr, 914 op/s 2024-05-08T22:00:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 350.0 s, 2060.8 tps, lat 46.124 ms stddev 53.659 2024-05-08T22:00:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:41 smithi160 ceph-mon[46962]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 955 op/s 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:41 smithi186 ceph-mon[46619]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 955 op/s 2024-05-08T22:00:42.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:41 smithi064 ceph-mon[42500]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 955 op/s 2024-05-08T22:00:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:43 smithi160 ceph-mon[46962]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-05-08T22:00:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:43 smithi186 ceph-mon[46619]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-05-08T22:00:44.223 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:43 smithi064 ceph-mon[42500]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-05-08T22:00:46.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:45 smithi160 ceph-mon[46962]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 1.74k op/s 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:45 smithi186 ceph-mon[46619]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 1.74k op/s 2024-05-08T22:00:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:45 smithi064 ceph-mon[42500]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 1.74k op/s 2024-05-08T22:00:46.481 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 355.0 s, 1927.4 tps, lat 54.302 ms stddev 96.308 2024-05-08T22:00:48.053 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:47 smithi160 ceph-mon[46962]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 1.86k op/s 2024-05-08T22:00:48.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:47 smithi186 ceph-mon[46619]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 1.86k op/s 2024-05-08T22:00:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:47 smithi064 ceph-mon[42500]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 1.86k op/s 2024-05-08T22:00:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.381+0000 7fd29e325700 1 -- 172.21.15.64:0/1276590390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2981090d0 msgr2=0x7fd29810f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.381+0000 7fd29e325700 1 --2- 172.21.15.64:0/1276590390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2981090d0 0x7fd29810f980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd2880099c0 tx=0x7fd288009cd0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 -- 172.21.15.64:0/1276590390 shutdown_connections 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 --2- 172.21.15.64:0/1276590390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd298107540 0x7fd298107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 --2- 172.21.15.64:0/1276590390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2981090d0 0x7fd29810f980 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 --2- 172.21.15.64:0/1276590390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298108730 0x7fd298108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 -- 172.21.15.64:0/1276590390 >> 172.21.15.64:0/1276590390 conn(0x7fd2980764d0 msgr2=0x7fd298078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 -- 172.21.15.64:0/1276590390 shutdown_connections 2024-05-08T22:00:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.384+0000 7fd29e325700 1 -- 172.21.15.64:0/1276590390 wait complete. 2024-05-08T22:00:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.385+0000 7fd29e325700 1 Processor -- start 2024-05-08T22:00:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.385+0000 7fd29e325700 1 -- start start 2024-05-08T22:00:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29e325700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 0x7fd29806f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29e325700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd298108730 0x7fd29806fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29e325700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2981090d0 0x7fd2981ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29e325700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298072dc0 con 0x7fd298108730 2024-05-08T22:00:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29e325700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298072f30 con 0x7fd2981090d0 2024-05-08T22:00:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29e325700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981adbc0 con 0x7fd298107540 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 0x7fd29806f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 0x7fd29806f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53310/0 (socket says 172.21.15.64:53310) 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 -- 172.21.15.64:0/3604301067 learned_addr learned my addr 172.21.15.64:0/3604301067 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd2977fe700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd298108730 0x7fd29806fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 -- 172.21.15.64:0/3604301067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2981090d0 msgr2=0x7fd2981ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2981090d0 0x7fd2981ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2981090d0 0x7fd2981ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 -- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd298108730 msgr2=0x7fd29806fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd298108730 0x7fd29806fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.386+0000 7fd297fff700 1 -- 172.21.15.64:0/3604301067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009520 con 0x7fd298107540 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.387+0000 7fd2977fe700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd298108730 0x7fd29806fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.387+0000 7fd297fff700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 0x7fd29806f7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00ea50 tx=0x7fd28c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.387+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2981090d0 0x7fd2981ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:00:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.387+0000 7fd2957fa700 1 -- 172.21.15.64:0/3604301067 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00cbd0 con 0x7fd298107540 2024-05-08T22:00:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.388+0000 7fd2957fa700 1 -- 172.21.15.64:0/3604301067 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd28c00cd30 con 0x7fd298107540 2024-05-08T22:00:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.388+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2880095f0 con 0x7fd298107540 2024-05-08T22:00:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.388+0000 7fd2957fa700 1 -- 172.21.15.64:0/3604301067 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c018710 con 0x7fd298107540 2024-05-08T22:00:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.388+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981ae050 con 0x7fd298107540 2024-05-08T22:00:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.393+0000 7fd2957fa700 1 -- 172.21.15.64:0/3604301067 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd28c018870 con 0x7fd298107540 2024-05-08T22:00:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.393+0000 7fd2aa4dfb80 1 Processor -- start 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 -- start start 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683ca80 0x55cb3683cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3683dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb367940f0 con 0x55cb3683d3e0 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36791ab0 con 0x55cb3683e2d0 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3678d3a0 con 0x55cb3683ca80 2024-05-08T22:00:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2957fa700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2800a1270 0x7fd2800a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd2977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd297fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3683dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd297fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3683dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37124/0 (socket says 172.21.15.64:37124) 2024-05-08T22:00:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.394+0000 7fd297fff700 1 -- 172.21.15.64:0/2026553600 learned_addr learned my addr 172.21.15.64:0/2026553600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd2957fa700 1 -- 172.21.15.64:0/3604301067 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd28c014070 con 0x7fd298107540 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683ca80 0x55cb3683cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd2977fe700 1 -- 172.21.15.64:0/2026553600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683ca80 msgr2=0x55cb3683cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd2977fe700 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683ca80 0x55cb3683cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd2977fe700 1 -- 172.21.15.64:0/2026553600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 msgr2=0x55cb3683dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd2977fe700 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3683dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.395+0000 7fd2977fe700 1 -- 172.21.15.64:0/2026553600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3660e350 con 0x55cb3683e2d0 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.396+0000 7fd2977fe700 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd29000b810 tx=0x7fd2900096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f441eb0656561010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.396+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2800a1270 0x7fd2800a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.396+0000 7fd27e7fc700 1 -- 172.21.15.64:0/2026553600 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290010040 con 0x55cb3683e2d0 2024-05-08T22:00:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.396+0000 7fd27e7fc700 1 -- 172.21.15.64:0/2026553600 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd29000ae30 con 0x55cb3683e2d0 2024-05-08T22:00:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.396+0000 7fd27e7fc700 1 -- 172.21.15.64:0/2026553600 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900135e0 con 0x55cb3683e2d0 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.397+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/2026553600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 msgr2=0x55cb3683e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.397+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd29000b810 tx=0x7fd2900096b0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/2026553600 shutdown_connections 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683ca80 0x55cb3683cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/2026553600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3683dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/2026553600 >> 172.21.15.64:0/2026553600 conn(0x55cb3679b1b0 msgr2=0x55cb36839440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2800a1270 0x7fd2800a3730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c010 tx=0x7fd288005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/2026553600 shutdown_connections 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/2026553600 wait complete. 2024-05-08T22:00:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.398+0000 7fd2aa4dfb80 1 -- start start 2024-05-08T22:00:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 Processor -- start 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb368468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb36846de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36729500 con 0x55cb3683d3e0 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb367291a0 con 0x55cb3683ca80 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36610640 con 0x55cb3683e2d0 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2aa4dfb80 10 client.0 _ll_register_callbacks cb 0x55cb36732ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb36846de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb36846de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37134/0 (socket says 172.21.15.64:37134) 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd2977fe700 1 -- 172.21.15.64:0/1302853694 learned_addr learned my addr 172.21.15.64:0/1302853694 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.399+0000 7fd297fff700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb368468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.400+0000 7fd29c8c2700 1 -- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 msgr2=0x55cb368468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.400+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb368468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.400+0000 7fd29c8c2700 1 -- 172.21.15.64:0/1302853694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 msgr2=0x55cb36846de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.400+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb36846de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.400+0000 7fd29c8c2700 1 -- 172.21.15.64:0/1302853694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29000b470 con 0x55cb3683e2d0 2024-05-08T22:00:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.400+0000 7fd2977fe700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb36846de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.402+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684de20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b580 tx=0x7fd288009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff239273743d47bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.402+0000 7fd2777fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:00:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.403+0000 7fd2777fe700 1 -- 172.21.15.64:0/1302853694 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28802a070 con 0x55cb3683e2d0 2024-05-08T22:00:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.403+0000 7fd2777fe700 1 -- 172.21.15.64:0/1302853694 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd288003d70 con 0x55cb3683e2d0 2024-05-08T22:00:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.403+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb366e6210 con 0x55cb3683e2d0 2024-05-08T22:00:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.403+0000 7fd2777fe700 1 -- 172.21.15.64:0/1302853694 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28801e070 con 0x55cb3683e2d0 2024-05-08T22:00:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.403+0000 7fd2aa4dfb80 10 client.25360 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.403+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55cb365bab00 con 0x55cb3683e2d0 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2777fe700 1 -- 172.21.15.64:0/1302853694 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb366e6210 con 0x55cb3683e2d0 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2777fe700 1 -- 172.21.15.64:0/1302853694 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd28800f460 con 0x55cb3683e2d0 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2aa4dfb80 10 client.25360 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2aa4dfb80 10 client.25360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2aa4dfb80 20 client.25360 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb36852680 0x55cb36854aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2aa4dfb80 4 client.25360 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:00:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.404+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb3680c7b0 con 0x55cb36852680 2024-05-08T22:00:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.405+0000 7fd297fff700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb36852680 0x55cb36854aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.405+0000 7fd297fff700 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb36852680 0x55cb36854aa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc3a94e3033e46a server_cookie=fcceb846dc116efa in_seq=0 out_seq=0 2024-05-08T22:00:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.405+0000 7fd2777fe700 10 client.25360 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.407+0000 7fd2777fe700 1 -- 172.21.15.64:0/1302853694 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb3680c7b0 con 0x55cb36852680 2024-05-08T22:00:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.407+0000 7fd2777fe700 10 client.25360 handle_command_reply: tid=0 2024-05-08T22:00:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.407+0000 7fd2aa4dfb80 1 client.25360 shutdown 2024-05-08T22:00:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 msgr2=0x55cb3684de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684de20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b580 tx=0x7fd288009310 comp rx=0 tx=0).stop 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 shutdown_connections 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb368468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb36852680 0x55cb36854aa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684de20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1302853694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb36846de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 >> 172.21.15.64:0/1302853694 conn(0x55cb3679b1b0 msgr2=0x55cb36837860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2777fe700 0 client.25360 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.408+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 shutdown_connections 2024-05-08T22:00:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.409+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1302853694 wait complete. 2024-05-08T22:00:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.409+0000 7fd2aa4dfb80 20 client.25360 trim_cache size 0 max 16384 2024-05-08T22:00:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 Processor -- start 2024-05-08T22:00:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 -- start start 2024-05-08T22:00:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683ca80 0x55cb36854a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683d3e0 0x55cb3684dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3684efa0 con 0x55cb3683ca80 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36846410 con 0x55cb3683e2d0 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.429+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36838930 con 0x55cb3683d3e0 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.430+0000 7fd29c8c2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.430+0000 7fd29c8c2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:58106/0 (socket says 172.21.15.64:58106) 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.430+0000 7fd29c8c2700 1 -- 172.21.15.64:0/1394543076 learned_addr learned my addr 172.21.15.64:0/1394543076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.430+0000 7fd2977fe700 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683d3e0 0x55cb3684dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.431+0000 7fd2977fe700 1 -- 172.21.15.64:0/1394543076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 msgr2=0x55cb3683e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.431+0000 7fd2977fe700 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.431+0000 7fd2977fe700 1 -- 172.21.15.64:0/1394543076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683ca80 msgr2=0x55cb36854a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.431+0000 7fd2977fe700 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683ca80 0x55cb36854a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.431+0000 7fd2977fe700 1 -- 172.21.15.64:0/1394543076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb36753500 con 0x55cb3683d3e0 2024-05-08T22:00:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.431+0000 7fd2977fe700 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683d3e0 0x55cb3684dd40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd29000be00 tx=0x7fd290003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767099fddb5afd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.432+0000 7fd27effd700 1 -- 172.21.15.64:0/1394543076 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290010040 con 0x55cb3683d3e0 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.432+0000 7fd27effd700 1 -- 172.21.15.64:0/1394543076 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd29000ac30 con 0x55cb3683d3e0 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.432+0000 7fd27effd700 1 -- 172.21.15.64:0/1394543076 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290004910 con 0x55cb3683d3e0 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1394543076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683d3e0 msgr2=0x55cb3684dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683d3e0 0x55cb3684dd40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd29000be00 tx=0x7fd290003ac0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1394543076 shutdown_connections 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683e2d0 0x55cb3683e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683ca80 0x55cb36854a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/1394543076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683d3e0 0x55cb3684dd40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1394543076 >> 172.21.15.64:0/1394543076 conn(0x55cb3679b1b0 msgr2=0x55cb368593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1394543076 shutdown_connections 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/1394543076 wait complete. 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.433+0000 7fd2aa4dfb80 1 -- start start 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 Processor -- start 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb3684b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3684b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3684ca60 con 0x55cb3683d3e0 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3684cbd0 con 0x55cb3683ca80 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3684cd90 con 0x55cb3683e2d0 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd2aa4dfb80 10 client.0 _ll_register_callbacks cb 0x55cb36732ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd297fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb3684b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd297fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb3684b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:58118/0 (socket says 172.21.15.64:58118) 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.434+0000 7fd297fff700 1 -- 172.21.15.64:0/3602099315 learned_addr learned my addr 172.21.15.64:0/3602099315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:00:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd29c8c2700 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd297fff700 1 -- 172.21.15.64:0/3602099315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 msgr2=0x55cb3684c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd297fff700 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd297fff700 1 -- 172.21.15.64:0/3602099315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 msgr2=0x55cb3684b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd297fff700 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3684b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd297fff700 1 -- 172.21.15.64:0/3602099315 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb3660e350 con 0x55cb3683ca80 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd297fff700 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb3684b020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c5b0 tx=0x7fd28c0ca650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c642bcb775dfcf66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd2777fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd2777fe700 1 -- 172.21.15.64:0/3602099315 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c09df00 con 0x55cb3683ca80 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd2777fe700 1 -- 172.21.15.64:0/3602099315 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd28c0a2550 con 0x55cb3683ca80 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.435+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd29000ae30 con 0x55cb3683ca80 2024-05-08T22:00:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2777fe700 1 -- 172.21.15.64:0/3602099315 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28c09a0d0 con 0x55cb3683ca80 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 10 client.25466 fetch_fsmap learned FSMap version 39 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55cb36612c70 con 0x55cb3683ca80 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2777fe700 1 -- 172.21.15.64:0/3602099315 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd29000ae30 con 0x55cb3683ca80 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2777fe700 1 -- 172.21.15.64:0/3602099315 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd28c0ab070 con 0x55cb3683ca80 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 10 client.25466 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 10 client.25466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 20 client.25466 populate_metadata read hostname 'smithi064' 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb368a90e0 0x55cb368ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 4 client.25466 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb36785e60 con 0x55cb368a90e0 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.436+0000 7fd2977fe700 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb368a90e0 0x55cb368ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.437+0000 7fd2777fe700 10 client.25466 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.437+0000 7fd2977fe700 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb368a90e0 0x55cb368ab500 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7c68f219aa63d78 server_cookie=5cc2edfe10236cd1 in_seq=0 out_seq=0 2024-05-08T22:00:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.448+0000 7fd2777fe700 1 -- 172.21.15.64:0/3602099315 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb36785e60 con 0x55cb368a90e0 2024-05-08T22:00:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.448+0000 7fd2777fe700 10 client.25466 handle_command_reply: tid=0 2024-05-08T22:00:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.449+0000 7fd2aa4dfb80 1 client.25466 shutdown 2024-05-08T22:00:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.460+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 msgr2=0x55cb3684b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.460+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb3684b020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c5b0 tx=0x7fd28c0ca650 comp rx=0 tx=0).stop 2024-05-08T22:00:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.471+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 shutdown_connections 2024-05-08T22:00:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.471+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3683ca80 0x55cb3684b020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.471+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cb368a90e0 0x55cb368ab500 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.471+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb3683e2d0 0x55cb3684c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.471+0000 7fd2aa4dfb80 1 --2- 172.21.15.64:0/3602099315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb3683d3e0 0x55cb3684b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.471+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 >> 172.21.15.64:0/3602099315 conn(0x55cb3679b1b0 msgr2=0x55cb36839020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.478+0000 7fd2777fe700 0 client.25466 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:00:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.479+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 shutdown_connections 2024-05-08T22:00:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.480+0000 7fd2aa4dfb80 1 -- 172.21.15.64:0/3602099315 wait complete. 2024-05-08T22:00:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.483+0000 7fd2aa4dfb80 20 client.25466 trim_cache size 0 max 16384 2024-05-08T22:00:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:00:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:00:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:00:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:00:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.494+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2800a1270 msgr2=0x7fd2800a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.494+0000 7fd29e325700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2800a1270 0x7fd2800a3730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c010 tx=0x7fd288005040 comp rx=0 tx=0).stop 2024-05-08T22:00:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.494+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 msgr2=0x7fd29806f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.494+0000 7fd29e325700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 0x7fd29806f7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00ea50 tx=0x7fd28c00ed60 comp rx=0 tx=0).stop 2024-05-08T22:00:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.497+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 shutdown_connections 2024-05-08T22:00:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.497+0000 7fd29e325700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2800a1270 0x7fd2800a3730 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.497+0000 7fd29e325700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2981090d0 0x7fd2981ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.497+0000 7fd29e325700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd298107540 0x7fd29806f7f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.497+0000 7fd29e325700 1 --2- 172.21.15.64:0/3604301067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd298108730 0x7fd29806fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.497+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 >> 172.21.15.64:0/3604301067 conn(0x7fd2980764d0 msgr2=0x7fd29810ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.510+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 shutdown_connections 2024-05-08T22:00:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:00:49.518+0000 7fd29e325700 1 -- 172.21.15.64:0/3604301067 wait complete. 2024-05-08T22:00:50.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:49 smithi160 ceph-mon[46962]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2024-05-08T22:00:50.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:49 smithi186 ceph-mon[46619]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2024-05-08T22:00:50.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:49 smithi064 ceph-mon[42500]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2024-05-08T22:00:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 360.0 s, 2480.5 tps, lat 40.625 ms stddev 45.703 2024-05-08T22:00:52.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:51 smithi160 ceph-mon[46962]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 MiB/s wr, 1.93k op/s 2024-05-08T22:00:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:51 smithi186 ceph-mon[46619]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 MiB/s wr, 1.93k op/s 2024-05-08T22:00:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:51 smithi064 ceph-mon[42500]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 MiB/s wr, 1.93k op/s 2024-05-08T22:00:54.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:53 smithi186 ceph-mon[46619]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2024-05-08T22:00:54.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:53 smithi064 ceph-mon[42500]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2024-05-08T22:00:54.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:53 smithi160 ceph-mon[46962]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2024-05-08T22:00:56.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:55 smithi186 ceph-mon[46619]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.71k op/s 2024-05-08T22:00:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:55 smithi064 ceph-mon[42500]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.71k op/s 2024-05-08T22:00:56.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:55 smithi160 ceph-mon[46962]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.71k op/s 2024-05-08T22:00:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 365.0 s, 2258.2 tps, lat 44.349 ms stddev 54.673 2024-05-08T22:00:58.152 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:57 smithi186 ceph-mon[46619]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-05-08T22:00:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:57 smithi064 ceph-mon[42500]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-05-08T22:00:58.297 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:57 smithi160 ceph-mon[46962]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-05-08T22:01:00.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:00:59 smithi186 ceph-mon[46619]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 829 op/s 2024-05-08T22:01:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:00:59 smithi064 ceph-mon[42500]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 829 op/s 2024-05-08T22:01:00.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:00:59 smithi160 ceph-mon[46962]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 829 op/s 2024-05-08T22:01:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 370.0 s, 2178.4 tps, lat 45.556 ms stddev 54.769 2024-05-08T22:01:01.820 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:01 smithi186 ceph-mon[46619]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 864 op/s 2024-05-08T22:01:02.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:01 smithi064 ceph-mon[42500]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 864 op/s 2024-05-08T22:01:02.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:01 smithi160 ceph-mon[46962]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 864 op/s 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 -- 172.21.15.64:0/2442314164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb704107540 msgr2=0x7fb704107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 --2- 172.21.15.64:0/2442314164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb704107540 0x7fb704107960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009990 tx=0x7fb6f4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 -- 172.21.15.64:0/2442314164 shutdown_connections 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 --2- 172.21.15.64:0/2442314164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7041090d0 0x7fb70410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 --2- 172.21.15.64:0/2442314164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb704107540 0x7fb704107960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 --2- 172.21.15.64:0/2442314164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb704108730 0x7fb704108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.713+0000 7fb70aa14700 1 -- 172.21.15.64:0/2442314164 >> 172.21.15.64:0/2442314164 conn(0x7fb7040764d0 msgr2=0x7fb704078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.714+0000 7fb70aa14700 1 -- 172.21.15.64:0/2442314164 shutdown_connections 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.714+0000 7fb70aa14700 1 -- 172.21.15.64:0/2442314164 wait complete. 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.714+0000 7fb70aa14700 1 Processor -- start 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.714+0000 7fb70aa14700 1 -- start start 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb70aa14700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb704107540 0x7fb70419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb70aa14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704108730 0x7fb70419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb70aa14700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 0x7fb7041a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb70aa14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041a5070 con 0x7fb704108730 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb70aa14700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041a51b0 con 0x7fb704107540 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb70aa14700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041a5340 con 0x7fb7041090d0 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 0x7fb7041a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 0x7fb7041a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:59512/0 (socket says 172.21.15.64:59512) 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 -- 172.21.15.64:0/552077871 learned_addr learned my addr 172.21.15.64:0/552077871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb7037fe700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704108730 0x7fb70419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 -- 172.21.15.64:0/552077871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb704107540 msgr2=0x7fb70419d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:01:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb704107540 0x7fb70419d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 -- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704108730 msgr2=0x7fb70419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704108730 0x7fb70419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.715+0000 7fb708fb1700 1 -- 172.21.15.64:0/552077871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40095f0 con 0x7fb7041090d0 2024-05-08T22:01:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.716+0000 7fb708fb1700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 0x7fb7041a4aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b970 tx=0x7fb6fc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.716+0000 7fb7017fa700 1 -- 172.21.15.64:0/552077871 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00c930 con 0x7fb7041090d0 2024-05-08T22:01:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.716+0000 7fb7017fa700 1 -- 172.21.15.64:0/552077871 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc014440 con 0x7fb7041090d0 2024-05-08T22:01:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.716+0000 7fb7017fa700 1 -- 172.21.15.64:0/552077871 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0075d0 con 0x7fb7041090d0 2024-05-08T22:01:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.717+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7041a5620 con 0x7fb7041090d0 2024-05-08T22:01:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.717+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7041a5ab0 con 0x7fb7041090d0 2024-05-08T22:01:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.722+0000 7fb716bceb80 1 Processor -- start 2024-05-08T22:01:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 -- start start 2024-05-08T22:01:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ede7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 0x55806ede8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede92d0 0x55806ede9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ed3f0f0 con 0x55806ede83e0 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ed3cab0 con 0x55806ede7a80 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.725+0000 7fb716bceb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ed383a0 con 0x55806ede92d0 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 0x55806ede8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 0x55806ede8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43288/0 (socket says 172.21.15.64:43288) 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 -- 172.21.15.64:0/2761549602 learned_addr learned my addr 172.21.15.64:0/2761549602 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 -- 172.21.15.64:0/2761549602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede92d0 msgr2=0x55806ede9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede92d0 0x55806ede9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 -- 172.21.15.64:0/2761549602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 msgr2=0x55806ede7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ede7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb703fff700 1 -- 172.21.15.64:0/2761549602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806ebb9350 con 0x55806ede83e0 2024-05-08T22:01:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.726+0000 7fb7017fa700 1 -- 172.21.15.64:0/552077871 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb6fc0145b0 con 0x7fb7041090d0 2024-05-08T22:01:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.727+0000 7fb703fff700 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 0x55806ede8cf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4006010 tx=0x7fb6f4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f6f759c5fa0857 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.727+0000 7fb6ea7fc700 1 -- 172.21.15.64:0/2761549602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f401d070 con 0x55806ede83e0 2024-05-08T22:01:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.727+0000 7fb6ea7fc700 1 -- 172.21.15.64:0/2761549602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f400bdd0 con 0x55806ede83e0 2024-05-08T22:01:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.727+0000 7fb6ea7fc700 1 -- 172.21.15.64:0/2761549602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4021410 con 0x55806ede83e0 2024-05-08T22:01:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.727+0000 7fb7017fa700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6ec0a11f0 0x7fb6ec0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.728+0000 7fb7017fa700 1 -- 172.21.15.64:0/552077871 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6fc006e40 con 0x7fb7041090d0 2024-05-08T22:01:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.728+0000 7fb716bceb80 1 -- 172.21.15.64:0/2761549602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 msgr2=0x55806ede8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.728+0000 7fb716bceb80 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 0x55806ede8cf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4006010 tx=0x7fb6f4005ab0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.728+0000 7fb716bceb80 1 -- 172.21.15.64:0/2761549602 shutdown_connections 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.728+0000 7fb716bceb80 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ede7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede92d0 0x55806ede9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 --2- 172.21.15.64:0/2761549602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede83e0 0x55806ede8cf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 -- 172.21.15.64:0/2761549602 >> 172.21.15.64:0/2761549602 conn(0x55806ed461b0 msgr2=0x55806ede4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 -- 172.21.15.64:0/2761549602 shutdown_connections 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 -- 172.21.15.64:0/2761549602 wait complete. 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 -- start start 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.729+0000 7fb716bceb80 1 Processor -- start 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee02820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee02d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee09da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ecd4500 con 0x55806ede92d0 2024-05-08T22:01:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ecd41a0 con 0x55806ede7a80 2024-05-08T22:01:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ebbb640 con 0x55806ede83e0 2024-05-08T22:01:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb716bceb80 10 client.0 _ll_register_callbacks cb 0x55806ecddad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee09da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee09da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43300/0 (socket says 172.21.15.64:43300) 2024-05-08T22:01:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 -- 172.21.15.64:0/3146315302 learned_addr learned my addr 172.21.15.64:0/3146315302 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 -- 172.21.15.64:0/3146315302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 msgr2=0x55806ee02d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee02d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 -- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 msgr2=0x55806ee02820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee02820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.730+0000 7fb708fb1700 1 -- 172.21.15.64:0/3146315302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7041102d0 con 0x55806ede92d0 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.731+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee09da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc095650 tx=0x7fb6fc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c5f487435abe13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb6e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3146315302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a15d0 con 0x55806ede92d0 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3146315302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6fc098640 con 0x55806ede92d0 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55806ec91210 con 0x55806ede92d0 2024-05-08T22:01:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3146315302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6fc098800 con 0x55806ede92d0 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb716bceb80 10 client.15729 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.732+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55806eb65b00 con 0x55806ede92d0 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.733+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3146315302 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55806ec91210 con 0x55806ede92d0 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3146315302 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6fc0980b0 con 0x55806ede92d0 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb716bceb80 10 client.15729 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb716bceb80 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb716bceb80 20 client.15729 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee0e670 0x55806ee10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb716bceb80 4 client.15729 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:01:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.734+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55806edb77b0 con 0x55806ee0e670 2024-05-08T22:01:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.736+0000 7fb7037fe700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6ec0a11f0 0x7fb6ec0a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.739+0000 7fb703fff700 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee0e670 0x55806ee10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.739+0000 7fb7037fe700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6ec0a11f0 0x7fb6ec0a36b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55806ede8ab0 tx=0x7fb6f80047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.739+0000 7fb703fff700 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee0e670 0x55806ee10b00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb4dbf4eed95543 server_cookie=3196e2df8bb6a945 in_seq=0 out_seq=0 2024-05-08T22:01:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.739+0000 7fb6e37fe700 10 client.15729 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3146315302 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55806edb77b0 con 0x55806ee0e670 2024-05-08T22:01:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb6e37fe700 10 client.15729 handle_command_reply: tid=0 2024-05-08T22:01:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 client.15729 shutdown 2024-05-08T22:01:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 msgr2=0x55806ee09da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee09da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc095650 tx=0x7fb6fc00cd70 comp rx=0 tx=0).stop 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 shutdown_connections 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee02820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee0e670 0x55806ee10b00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee09da0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3146315302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee02d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb6e37fe700 0 client.15729 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 >> 172.21.15.64:0/3146315302 conn(0x55806ed461b0 msgr2=0x55806ede2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 shutdown_connections 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.742+0000 7fb716bceb80 1 -- 172.21.15.64:0/3146315302 wait complete. 2024-05-08T22:01:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.743+0000 7fb716bceb80 20 client.15729 trim_cache size 0 max 16384 2024-05-08T22:01:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 Processor -- start 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 -- start start 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ede96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ede7690 con 0x55806ede92d0 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ee03830 con 0x55806ede7a80 2024-05-08T22:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.769+0000 7fb716bceb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ee15d50 con 0x55806ede83e0 2024-05-08T22:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.776+0000 7fb703fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ede96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ede96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43310/0 (socket says 172.21.15.64:43310) 2024-05-08T22:01:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 -- 172.21.15.64:0/3113917213 learned_addr learned my addr 172.21.15.64:0/3113917213 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 -- 172.21.15.64:0/3113917213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 msgr2=0x55806ee09d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 -- 172.21.15.64:0/3113917213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 msgr2=0x55806ee10a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee10a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.777+0000 7fb708fb1700 1 -- 172.21.15.64:0/3113917213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806ecfe500 con 0x55806ede92d0 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.778+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ede96b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc098440 tx=0x7fb6fc09d100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b30fc326d411bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.778+0000 7fb6eaffd700 1 -- 172.21.15.64:0/3113917213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a15d0 con 0x55806ede92d0 2024-05-08T22:01:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.778+0000 7fb6eaffd700 1 -- 172.21.15.64:0/3113917213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc0980b0 con 0x55806ede92d0 2024-05-08T22:01:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.778+0000 7fb6eaffd700 1 -- 172.21.15.64:0/3113917213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a1140 con 0x55806ede92d0 2024-05-08T22:01:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.779+0000 7fb716bceb80 1 -- 172.21.15.64:0/3113917213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 msgr2=0x55806ede96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.779+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ede96b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc098440 tx=0x7fb6fc09d100 comp rx=0 tx=0).stop 2024-05-08T22:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.782+0000 7fb716bceb80 1 -- 172.21.15.64:0/3113917213 shutdown_connections 2024-05-08T22:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.782+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ede7a80 0x55806ee10a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.782+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ede96b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.782+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3113917213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.782+0000 7fb716bceb80 1 -- 172.21.15.64:0/3113917213 >> 172.21.15.64:0/3113917213 conn(0x55806ed461b0 msgr2=0x55806ee01d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.789+0000 7fb716bceb80 1 -- 172.21.15.64:0/3113917213 shutdown_connections 2024-05-08T22:01:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.796+0000 7fb716bceb80 1 -- 172.21.15.64:0/3113917213 wait complete. 2024-05-08T22:01:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.796+0000 7fb716bceb80 1 -- start start 2024-05-08T22:01:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.796+0000 7fb716bceb80 1 Processor -- start 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee04f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee05470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ee71c00 0x55806ee73ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ee05ba0 con 0x55806ede92d0 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ee74560 con 0x55806ee71c00 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ee74720 con 0x55806ede83e0 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb716bceb80 10 client.0 _ll_register_callbacks cb 0x55806ecddad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee04f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee04f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:59532/0 (socket says 172.21.15.64:59532) 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 -- 172.21.15.64:0/3633062306 learned_addr learned my addr 172.21.15.64:0/3633062306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb708fb1700 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ee71c00 0x55806ee73ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 -- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ee71c00 msgr2=0x55806ee73ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ee71c00 0x55806ee73ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 -- 172.21.15.64:0/3633062306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 msgr2=0x55806ee05470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee05470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.797+0000 7fb703fff700 1 -- 172.21.15.64:0/3633062306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806eb65b00 con 0x55806ede83e0 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.798+0000 7fb703fff700 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee04f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005e50 tx=0x7fb6f4005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5153810f3d652f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.798+0000 7fb6e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.799+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3633062306 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4033070 con 0x55806ede83e0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.799+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3633062306 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6f400bbe0 con 0x55806ede83e0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6f400bdd0 con 0x55806ede83e0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3633062306 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f401d040 con 0x55806ede83e0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 10 client.25375 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55806ebbdc70 con 0x55806ede83e0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3633062306 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f400bdd0 con 0x55806ede83e0 2024-05-08T22:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3633062306 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6f4036030 con 0x55806ede83e0 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 10 client.25375 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 10 client.25375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 20 client.25375 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee75d90 0x55806ee78180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 4 client.25375 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.800+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55806ed30e60 con 0x55806ee75d90 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.801+0000 7fb7037fe700 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee75d90 0x55806ee78180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.801+0000 7fb6e37fe700 10 client.25375 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.801+0000 7fb7037fe700 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee75d90 0x55806ee78180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b93b9c710ea573 server_cookie=6d48f920ee959ee2 in_seq=0 out_seq=0 2024-05-08T22:01:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.826+0000 7fb6e37fe700 1 -- 172.21.15.64:0/3633062306 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55806ed30e60 con 0x55806ee75d90 2024-05-08T22:01:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.826+0000 7fb6e37fe700 10 client.25375 handle_command_reply: tid=0 2024-05-08T22:01:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.826+0000 7fb716bceb80 1 client.25375 shutdown 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 msgr2=0x55806ee04f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee04f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005e50 tx=0x7fb6f4005c60 comp rx=0 tx=0).stop 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 shutdown_connections 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55806ee71c00 0x55806ee73ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55806ee75d90 0x55806ee78180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55806ede83e0 0x55806ee04f30 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005e50 tx=0x7fb6f4005c60 comp rx=0 tx=0).stop 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 --2- 172.21.15.64:0/3633062306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55806ede92d0 0x55806ee05470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.827+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 >> 172.21.15.64:0/3633062306 conn(0x55806ed461b0 msgr2=0x55806edef3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.828+0000 7fb6e37fe700 0 client.25375 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.828+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 shutdown_connections 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.828+0000 7fb716bceb80 1 -- 172.21.15.64:0/3633062306 wait complete. 2024-05-08T22:01:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.828+0000 7fb716bceb80 20 client.25375 trim_cache size 0 max 16384 2024-05-08T22:01:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:01:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:01:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:01:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6ec0a11f0 msgr2=0x7fb6ec0a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6ec0a11f0 0x7fb6ec0a36b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55806ede8ab0 tx=0x7fb6f80047c0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 msgr2=0x7fb7041a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 0x7fb7041a4aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b970 tx=0x7fb6fc00bc80 comp rx=0 tx=0).stop 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 shutdown_connections 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6ec0a11f0 0x7fb6ec0a36b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb704107540 0x7fb70419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7041090d0 0x7fb7041a4aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 --2- 172.21.15.64:0/552077871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704108730 0x7fb70419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 >> 172.21.15.64:0/552077871 conn(0x7fb7040764d0 msgr2=0x7fb70410bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 shutdown_connections 2024-05-08T22:01:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:02.829+0000 7fb70aa14700 1 -- 172.21.15.64:0/552077871 wait complete. 2024-05-08T22:01:04.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:03 smithi186 ceph-mon[46619]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-05-08T22:01:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:03 smithi064 ceph-mon[42500]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-05-08T22:01:04.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:03 smithi160 ceph-mon[46962]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-05-08T22:01:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:05 smithi186 ceph-mon[46619]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 844 op/s 2024-05-08T22:01:06.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:05 smithi064 ceph-mon[42500]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 844 op/s 2024-05-08T22:01:06.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:05 smithi160 ceph-mon[46962]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 844 op/s 2024-05-08T22:01:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 375.0 s, 2246.0 tps, lat 44.730 ms stddev 53.656 2024-05-08T22:01:08.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:07 smithi186 ceph-mon[46619]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-05-08T22:01:08.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:07 smithi064 ceph-mon[42500]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-05-08T22:01:08.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:07 smithi160 ceph-mon[46962]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-05-08T22:01:10.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:09 smithi186 ceph-mon[46619]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 755 op/s 2024-05-08T22:01:10.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:09 smithi064 ceph-mon[42500]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 755 op/s 2024-05-08T22:01:10.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:09 smithi160 ceph-mon[46962]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 755 op/s 2024-05-08T22:01:11.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 380.0 s, 2328.2 tps, lat 42.753 ms stddev 46.555 2024-05-08T22:01:12.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:11 smithi186 ceph-mon[46619]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 791 op/s 2024-05-08T22:01:12.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:11 smithi064 ceph-mon[42500]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 791 op/s 2024-05-08T22:01:12.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:11 smithi160 ceph-mon[46962]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 791 op/s 2024-05-08T22:01:14.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:13 smithi064 ceph-mon[42500]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-05-08T22:01:14.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:13 smithi160 ceph-mon[46962]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-05-08T22:01:14.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:13 smithi186 ceph-mon[46619]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-05-08T22:01:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/2052140234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68104370 msgr2=0x7fbf68106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/2052140234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68104370 0x7fbf68106760 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009990 tx=0x7fbf5c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/2052140234 shutdown_connections 2024-05-08T22:01:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/2052140234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf68106ca0 0x7fbf68109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/2052140234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf68101a40 0x7fbf68103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/2052140234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68104370 0x7fbf68106760 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/2052140234 >> 172.21.15.64:0/2052140234 conn(0x7fbf680fb3b0 msgr2=0x7fbf680fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/2052140234 shutdown_connections 2024-05-08T22:01:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:15.999+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/2052140234 wait complete. 2024-05-08T22:01:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.000+0000 7fbf6ea1a700 1 Processor -- start 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.000+0000 7fbf6ea1a700 1 -- start start 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6ea1a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf68101a40 0x7fbf68101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6ea1a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf68104370 0x7fbf680ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6ea1a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 0x7fbf680fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6ea1a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681a7260 con 0x7fbf68106ca0 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6ea1a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681a73d0 con 0x7fbf68101a40 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6ea1a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681a7590 con 0x7fbf68104370 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6cfb7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 0x7fbf680fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6cfb7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 0x7fbf680fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55942/0 (socket says 172.21.15.64:55942) 2024-05-08T22:01:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6cfb7700 1 -- 172.21.15.64:0/311186837 learned_addr learned my addr 172.21.15.64:0/311186837 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.001+0000 7fbf6cfb7700 1 -- 172.21.15.64:0/311186837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf68104370 msgr2=0x7fbf680ff9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6cfb7700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf68104370 0x7fbf680ff9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6cfb7700 1 -- 172.21.15.64:0/311186837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf68101a40 msgr2=0x7fbf68101360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6cfb7700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf68101a40 0x7fbf68101360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6cfb7700 1 -- 172.21.15.64:0/311186837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0095f0 con 0x7fbf68106ca0 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6cfb7700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 0x7fbf680fff30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000ef30 tx=0x7fbf6000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf657fa700 1 -- 172.21.15.64:0/311186837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60018070 con 0x7fbf68106ca0 2024-05-08T22:01:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf657fa700 1 -- 172.21.15.64:0/311186837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf6000f040 con 0x7fbf68106ca0 2024-05-08T22:01:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf657fa700 1 -- 172.21.15.64:0/311186837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003de0 con 0x7fbf68106ca0 2024-05-08T22:01:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf681a77b0 con 0x7fbf68106ca0 2024-05-08T22:01:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.002+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf681a7c70 con 0x7fbf68106ca0 2024-05-08T22:01:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.006+0000 7fbf7abd4b80 1 Processor -- start 2024-05-08T22:01:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.007+0000 7fbf657fa700 1 -- 172.21.15.64:0/311186837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbf60004ad0 con 0x7fbf68106ca0 2024-05-08T22:01:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.007+0000 7fbf7abd4b80 1 -- start start 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5a9aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9b3e0 0x5595d5a9bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5a9c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d59f20f0 con 0x5595d5a9b3e0 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d59efab0 con 0x5595d5a9c2d0 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d59eb3a0 con 0x5595d5a9aa80 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5a9aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5a9aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:59464/0 (socket says 172.21.15.64:59464) 2024-05-08T22:01:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf677fe700 1 -- 172.21.15.64:0/3066369260 learned_addr learned my addr 172.21.15.64:0/3066369260 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5a9c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf657fa700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbf500a11a0 0x7fbf500a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf67fff700 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9b3e0 0x5595d5a9bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.008+0000 7fbf677fe700 1 -- 172.21.15.64:0/3066369260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 msgr2=0x5595d5a9c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5a9c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf677fe700 1 -- 172.21.15.64:0/3066369260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9b3e0 msgr2=0x5595d5a9bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9b3e0 0x5595d5a9bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf677fe700 1 -- 172.21.15.64:0/3066369260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d586c350 con 0x5595d5a9aa80 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf657fa700 1 -- 172.21.15.64:0/311186837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf600cbca0 con 0x7fbf68106ca0 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5a9aea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0098e0 tx=0x7fbf5c00b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e7629112535a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf6cfb7700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbf500a11a0 0x7fbf500a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf4e7fc700 1 -- 172.21.15.64:0/3066369260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c01b070 con 0x5595d5a9aa80 2024-05-08T22:01:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf4e7fc700 1 -- 172.21.15.64:0/3066369260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5c005a00 con 0x5595d5a9aa80 2024-05-08T22:01:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.009+0000 7fbf4e7fc700 1 -- 172.21.15.64:0/3066369260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c005020 con 0x5595d5a9aa80 2024-05-08T22:01:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf6cfb7700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbf500a11a0 0x7fbf500a3660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009c10 tx=0x7fbf60094d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3066369260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 msgr2=0x5595d5a9aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5a9aea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0098e0 tx=0x7fbf5c00b770 comp rx=0 tx=0).stop 2024-05-08T22:01:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3066369260 shutdown_connections 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5a9c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5a9aea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3066369260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9b3e0 0x5595d5a9bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3066369260 >> 172.21.15.64:0/3066369260 conn(0x5595d59f91b0 msgr2=0x5595d5a97440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3066369260 shutdown_connections 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.010+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3066369260 wait complete. 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 -- start start 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 Processor -- start 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5aa4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5987500 con 0x5595d5a9c2d0 2024-05-08T22:01:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d59871a0 con 0x5595d5a9b3e0 2024-05-08T22:01:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d586e640 con 0x5595d5a9aa80 2024-05-08T22:01:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf7abd4b80 10 client.0 _ll_register_callbacks cb 0x5595d5990ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.011+0000 7fbf67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:38768/0 (socket says 172.21.15.64:38768) 2024-05-08T22:01:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf67fff700 1 -- 172.21.15.64:0/1248891574 learned_addr learned my addr 172.21.15.64:0/1248891574 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5aa4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 -- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9b3e0 msgr2=0x5595d5aa4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 -- 172.21.15.64:0/1248891574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 msgr2=0x5595d5aabe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aabe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 -- 172.21.15.64:0/1248891574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf68109c80 con 0x5595d5a9aa80 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.012+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5aa4890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005fd0 tx=0x7fbf5c005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52c9ee8f43a9581 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf477fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf477fe700 1 -- 172.21.15.64:0/1248891574 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c01b070 con 0x5595d5a9aa80 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf477fe700 1 -- 172.21.15.64:0/1248891574 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf5c00b8b0 con 0x5595d5a9aa80 2024-05-08T22:01:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595d5944210 con 0x5595d5a9aa80 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf477fe700 1 -- 172.21.15.64:0/1248891574 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf5c00ba70 con 0x5595d5a9aa80 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 10 client.25384 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5595d5818b00 con 0x5595d5a9aa80 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf477fe700 1 -- 172.21.15.64:0/1248891574 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595d5944210 con 0x5595d5a9aa80 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf477fe700 1 -- 172.21.15.64:0/1248891574 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbf5c003680 con 0x5595d5a9aa80 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 20 client.25384 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5ab01e0 0x5595d5ab25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 4 client.25384 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.013+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595d5a6a7b0 con 0x5595d5ab01e0 2024-05-08T22:01:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.014+0000 7fbf67fff700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5ab01e0 0x5595d5ab25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.016+0000 7fbf477fe700 10 client.25384 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.016+0000 7fbf67fff700 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5ab01e0 0x5595d5ab25d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4137bae45ef75f8 server_cookie=6799916d87d0c4aa in_seq=0 out_seq=0 2024-05-08T22:01:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.017+0000 7fbf477fe700 1 -- 172.21.15.64:0/1248891574 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595d5a6a7b0 con 0x5595d5ab01e0 2024-05-08T22:01:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.017+0000 7fbf477fe700 10 client.25384 handle_command_reply: tid=0 2024-05-08T22:01:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.017+0000 7fbf7abd4b80 1 client.25384 shutdown 2024-05-08T22:01:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.018+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 msgr2=0x5595d5aa4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.018+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5aa4890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005fd0 tx=0x7fbf5c005bc0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 shutdown_connections 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5ab01e0 0x5595d5ab25d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aabe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9aa80 0x5595d5aa4890 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1248891574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 >> 172.21.15.64:0/1248891574 conn(0x5595d59f91b0 msgr2=0x5595d5a95860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf477fe700 0 client.25384 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 shutdown_connections 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1248891574 wait complete. 2024-05-08T22:01:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.019+0000 7fbf7abd4b80 20 client.25384 trim_cache size 0 max 16384 2024-05-08T22:01:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 Processor -- start 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 -- start start 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9aa80 0x5595d5a9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5a90330 con 0x5595d5a9c2d0 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5a9c150 con 0x5595d5a9aa80 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5aacf90 con 0x5595d5a9b3e0 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55962/0 (socket says 172.21.15.64:55962) 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf677fe700 1 -- 172.21.15.64:0/1390971424 learned_addr learned my addr 172.21.15.64:0/1390971424 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.040+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9aa80 0x5595d5a9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf67fff700 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aabdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf677fe700 1 -- 172.21.15.64:0/1390971424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 msgr2=0x5595d5aabdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aabdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf677fe700 1 -- 172.21.15.64:0/1390971424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9aa80 msgr2=0x5595d5a9c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9aa80 0x5595d5a9c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf677fe700 1 -- 172.21.15.64:0/1390971424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d59b1500 con 0x5595d5a9c2d0 2024-05-08T22:01:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.041+0000 7fbf677fe700 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa2890 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x5595d5a9b890 tx=0x7fbf5c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191063f421e35d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.042+0000 7fbf4effd700 1 -- 172.21.15.64:0/1390971424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c01b070 con 0x5595d5a9c2d0 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.042+0000 7fbf4effd700 1 -- 172.21.15.64:0/1390971424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5c01c040 con 0x5595d5a9c2d0 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.042+0000 7fbf4effd700 1 -- 172.21.15.64:0/1390971424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c027630 con 0x5595d5a9c2d0 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1390971424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 msgr2=0x5595d5aa2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa2890 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x5595d5a9b890 tx=0x7fbf5c005c30 comp rx=0 tx=0).stop 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1390971424 shutdown_connections 2024-05-08T22:01:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9aa80 0x5595d5a9c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa2890 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/1390971424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aabdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1390971424 >> 172.21.15.64:0/1390971424 conn(0x5595d59f91b0 msgr2=0x5595d5aa3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1390971424 shutdown_connections 2024-05-08T22:01:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.043+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/1390971424 wait complete. 2024-05-08T22:01:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 -- start start 2024-05-08T22:01:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 Processor -- start 2024-05-08T22:01:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 0x5595d5aabdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5b04690 con 0x5595d5a9aa80 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5b047d0 con 0x5595d5a9c2d0 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d5b04960 con 0x5595d5a9b3e0 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf7abd4b80 10 client.0 _ll_register_callbacks cb 0x5595d5990ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 0x5595d5aabdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 0x5595d5aabdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55974/0 (socket says 172.21.15.64:55974) 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf677fe700 1 -- 172.21.15.64:0/3043262781 learned_addr learned my addr 172.21.15.64:0/3043262781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.044+0000 7fbf67fff700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 -- 172.21.15.64:0/3043262781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 msgr2=0x5595d5aa8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 -- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 msgr2=0x5595d5aa8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 -- 172.21.15.64:0/3043262781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595d5818b00 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf677fe700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 0x5595d5aabdc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009ce0 tx=0x7fbf5c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bf846b0c20b577 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf477fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf477fe700 1 -- 172.21.15.64:0/3043262781 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c01b040 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf477fe700 1 -- 172.21.15.64:0/3043262781 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf5c000e40 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf5c005a00 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf477fe700 1 -- 172.21.15.64:0/3043262781 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf5c003cb0 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf7abd4b80 10 client.15750 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.045+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5595d5870c70 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf477fe700 1 -- 172.21.15.64:0/3043262781 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5c005a00 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf477fe700 1 -- 172.21.15.64:0/3043262781 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbf5c0040b0 con 0x5595d5a9aa80 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf7abd4b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf7abd4b80 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf7abd4b80 20 client.15750 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5b05ee0 0x5595d5b082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf7abd4b80 4 client.15750 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595d59e3e60 con 0x5595d5b05ee0 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.046+0000 7fbf67fff700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5b05ee0 0x5595d5b082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.047+0000 7fbf477fe700 10 client.15750 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.047+0000 7fbf67fff700 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5b05ee0 0x5595d5b082d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd341daa5c42a8a5 server_cookie=7094bd7127f15163 in_seq=0 out_seq=0 2024-05-08T22:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.109+0000 7fbf477fe700 1 -- 172.21.15.64:0/3043262781 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595d59e3e60 con 0x5595d5b05ee0 2024-05-08T22:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.109+0000 7fbf477fe700 10 client.15750 handle_command_reply: tid=0 2024-05-08T22:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.109+0000 7fbf7abd4b80 1 client.15750 shutdown 2024-05-08T22:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.109+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 msgr2=0x5595d5aabdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.109+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 0x5595d5aabdc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009ce0 tx=0x7fbf5c003680 comp rx=0 tx=0).stop 2024-05-08T22:01:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 shutdown_connections 2024-05-08T22:01:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595d5a9c2d0 0x5595d5aa8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5595d5b05ee0 0x5595d5b082d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595d5a9aa80 0x5595d5aabdc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 --2- 172.21.15.64:0/3043262781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595d5a9b3e0 0x5595d5aa8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 >> 172.21.15.64:0/3043262781 conn(0x5595d59f91b0 msgr2=0x5595d5ab21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf477fe700 0 client.15750 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 shutdown_connections 2024-05-08T22:01:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.111+0000 7fbf7abd4b80 1 -- 172.21.15.64:0/3043262781 wait complete. 2024-05-08T22:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.112+0000 7fbf7abd4b80 20 client.15750 trim_cache size 0 max 16384 2024-05-08T22:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.112+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbf500a11a0 msgr2=0x7fbf500a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.112+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbf500a11a0 0x7fbf500a3660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009c10 tx=0x7fbf60094d20 comp rx=0 tx=0).stop 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.112+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 msgr2=0x7fbf680fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.112+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 0x7fbf680fff30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000ef30 tx=0x7fbf6000c450 comp rx=0 tx=0).stop 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 shutdown_connections 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbf500a11a0 0x7fbf500a3660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf68101a40 0x7fbf68101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf68106ca0 0x7fbf680fff30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 --2- 172.21.15.64:0/311186837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf68104370 0x7fbf680ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 >> 172.21.15.64:0/311186837 conn(0x7fbf680fb3b0 msgr2=0x7fbf681038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 shutdown_connections 2024-05-08T22:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:16.113+0000 7fbf6ea1a700 1 -- 172.21.15.64:0/311186837 wait complete. 2024-05-08T22:01:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:15 smithi064 ceph-mon[42500]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 824 op/s 2024-05-08T22:01:16.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:15 smithi160 ceph-mon[46962]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 824 op/s 2024-05-08T22:01:16.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:15 smithi186 ceph-mon[46619]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 824 op/s 2024-05-08T22:01:16.485 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 385.0 s, 1504.6 tps, lat 66.388 ms stddev 77.761 2024-05-08T22:01:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:17 smithi064 ceph-mon[42500]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 737 op/s 2024-05-08T22:01:18.300 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:17 smithi160 ceph-mon[46962]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 737 op/s 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:17 smithi186 ceph-mon[46619]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 737 op/s 2024-05-08T22:01:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:19 smithi064 ceph-mon[42500]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 761 op/s 2024-05-08T22:01:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:19 smithi160 ceph-mon[46962]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 761 op/s 2024-05-08T22:01:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:19 smithi186 ceph-mon[46619]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 761 op/s 2024-05-08T22:01:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 390.0 s, 2044.1 tps, lat 48.921 ms stddev 54.084 2024-05-08T22:01:22.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:21 smithi160 ceph-mon[46962]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 862 op/s 2024-05-08T22:01:22.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:21 smithi186 ceph-mon[46619]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 862 op/s 2024-05-08T22:01:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:21 smithi064 ceph-mon[42500]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 862 op/s 2024-05-08T22:01:24.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:24 smithi160 ceph-mon[46962]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 822 op/s 2024-05-08T22:01:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:24 smithi186 ceph-mon[46619]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 822 op/s 2024-05-08T22:01:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:24 smithi064 ceph-mon[42500]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 822 op/s 2024-05-08T22:01:26.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:26 smithi160 ceph-mon[46962]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 859 op/s 2024-05-08T22:01:26.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:26 smithi186 ceph-mon[46619]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 859 op/s 2024-05-08T22:01:26.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:26 smithi064 ceph-mon[42500]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 859 op/s 2024-05-08T22:01:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 395.0 s, 1995.8 tps, lat 50.025 ms stddev 57.651 2024-05-08T22:01:28.398 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:28 smithi186 ceph-mon[46619]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 805 op/s 2024-05-08T22:01:28.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:28 smithi064 ceph-mon[42500]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 805 op/s 2024-05-08T22:01:28.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:28 smithi160 ceph-mon[46962]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 805 op/s 2024-05-08T22:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 -- 172.21.15.64:0/1505757488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b040ffe00 msgr2=0x7f6b04100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 --2- 172.21.15.64:0/1505757488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b040ffe00 0x7f6b04100220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009920 tx=0x7f6af4009c30 comp rx=0 tx=0).stop 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 -- 172.21.15.64:0/1505757488 shutdown_connections 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 --2- 172.21.15.64:0/1505757488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b040ffe00 0x7f6b04100220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 --2- 172.21.15.64:0/1505757488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b04101990 0x7f6b04108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 --2- 172.21.15.64:0/1505757488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b04100ff0 0x7f6b04101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 -- 172.21.15.64:0/1505757488 >> 172.21.15.64:0/1505757488 conn(0x7f6b040fb390 msgr2=0x7f6b040fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 -- 172.21.15.64:0/1505757488 shutdown_connections 2024-05-08T22:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.289+0000 7f6b09fef700 1 -- 172.21.15.64:0/1505757488 wait complete. 2024-05-08T22:01:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.290+0000 7f6b09fef700 1 Processor -- start 2024-05-08T22:01:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 -- start start 2024-05-08T22:01:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b040ffe00 0x7f6b041991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 0x7f6b04199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b04101990 0x7f6b041a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041a0d00 con 0x7f6b040ffe00 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041a0e40 con 0x7f6b04100ff0 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b09fef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041a0fd0 con 0x7f6b04101990 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 0x7f6b04199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b040ffe00 0x7f6b041991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 0x7f6b04199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54406/0 (socket says 172.21.15.64:54406) 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 -- 172.21.15.64:0/765821095 learned_addr learned my addr 172.21.15.64:0/765821095 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b03fff700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b04101990 0x7f6b041a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 -- 172.21.15.64:0/765821095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b04101990 msgr2=0x7f6b041a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b04101990 0x7f6b041a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 -- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b040ffe00 msgr2=0x7f6b041991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b040ffe00 0x7f6b041991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b02ffd700 1 -- 172.21.15.64:0/765821095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc009d00 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.291+0000 7f6b037fe700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b040ffe00 0x7f6b041991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.292+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 0x7f6b04199710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6af800d950 tx=0x7f6af800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.292+0000 7f6b00ff9700 1 -- 172.21.15.64:0/765821095 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800f890 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.292+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af4009520 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.292+0000 7f6b00ff9700 1 -- 172.21.15.64:0/765821095 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af8010460 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.292+0000 7f6b00ff9700 1 -- 172.21.15.64:0/765821095 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800e610 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.292+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b041a1630 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.294+0000 7f6b00ff9700 1 -- 172.21.15.64:0/765821095 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6af80108d0 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.295+0000 7f6b00ff9700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6aec0a1170 0x7f6aec0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.295+0000 7f6b00ff9700 1 -- 172.21.15.64:0/765821095 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6af80cc690 con 0x7f6b04100ff0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.295+0000 7f6b037fe700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6aec0a1170 0x7f6aec0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.295+0000 7f6b037fe700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6aec0a1170 0x7f6aec0a3630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005950 tx=0x7f6af4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 Processor -- start 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 -- start start 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e2f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e2ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7d7d0f0 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7d7aab0 con 0x55f1b7e2e290 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7d763a0 con 0x55f1b7e2fae0 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54414/0 (socket says 172.21.15.64:54414) 2024-05-08T22:01:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.296+0000 7f6b037fe700 1 -- 172.21.15.64:0/1475819946 learned_addr learned my addr 172.21.15.64:0/1475819946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e2ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e2f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 -- 172.21.15.64:0/1475819946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2e290 msgr2=0x55f1b7e2e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 -- 172.21.15.64:0/1475819946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 msgr2=0x55f1b7e2f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e2f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 -- 172.21.15.64:0/1475819946 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1b7bf7350 con 0x55f1b7e2fae0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6b03fff700 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e2ff60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009cd0 tx=0x7f6afc0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aed16a19b1fdb4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.297+0000 7f6ae9ffb700 1 -- 172.21.15.64:0/1475819946 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00e030 con 0x55f1b7e2fae0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.298+0000 7f6ae9ffb700 1 -- 172.21.15.64:0/1475819946 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc00f040 con 0x55f1b7e2fae0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.298+0000 7f6ae9ffb700 1 -- 172.21.15.64:0/1475819946 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc005280 con 0x55f1b7e2fae0 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.298+0000 7f6b161a9b80 1 -- 172.21.15.64:0/1475819946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 msgr2=0x55f1b7e2ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.298+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e2ff60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009cd0 tx=0x7f6afc0099e0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 -- 172.21.15.64:0/1475819946 shutdown_connections 2024-05-08T22:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e2ff60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/1475819946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e2f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 -- 172.21.15.64:0/1475819946 >> 172.21.15.64:0/1475819946 conn(0x55f1b7d841b0 msgr2=0x55f1b7e22440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 -- 172.21.15.64:0/1475819946 shutdown_connections 2024-05-08T22:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 -- 172.21.15.64:0/1475819946 wait complete. 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 -- start start 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.299+0000 7f6b161a9b80 1 Processor -- start 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e49590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7d12500 con 0x55f1b7e2fae0 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7d121a0 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7bf9640 con 0x55f1b7e2e290 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b161a9b80 10 client.0 _ll_register_callbacks cb 0x55f1b7d1bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e49050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56034/0 (socket says 172.21.15.64:56034) 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b037fe700 1 -- 172.21.15.64:0/192104089 learned_addr learned my addr 172.21.15.64:0/192104089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.300+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e49590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6b037fe700 1 -- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 msgr2=0x55f1b7e49590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6b037fe700 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e49590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6b037fe700 1 -- 172.21.15.64:0/192104089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2fae0 msgr2=0x55f1b7e505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6b037fe700 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6b037fe700 1 -- 172.21.15.64:0/192104089 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc009160 con 0x55f1b7e2e290 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6b037fe700 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e49050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c010 tx=0x7f6af4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6955222d031a9c4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.301+0000 7f6ae2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.302+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/192104089 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4003680 con 0x55f1b7e2e290 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.302+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/192104089 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6af40037e0 con 0x55f1b7e2e290 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.302+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1b7ccf210 con 0x55f1b7e2e290 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.302+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/192104089 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af40039a0 con 0x55f1b7e2e290 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 10 client.25402 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f1b7ba3b00 con 0x55f1b7e2e290 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/192104089 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1b7ccf210 con 0x55f1b7e2e290 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/192104089 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6af4037070 con 0x55f1b7e2e290 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 10 client.25402 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 10 client.25402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 20 client.25402 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7e27240 0x55f1b7e29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 4 client.25402 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.303+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1b7df57b0 con 0x55f1b7e27240 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.304+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7e27240 0x55f1b7e29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.304+0000 7f6ae2ffd700 10 client.25402 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.304+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7e27240 0x55f1b7e29660 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae60b582e4d940ed server_cookie=6fffcba8c52a14fb in_seq=0 out_seq=0 2024-05-08T22:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/192104089 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1b7df57b0 con 0x55f1b7e27240 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6ae2ffd700 10 client.25402 handle_command_reply: tid=0 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 client.25402 shutdown 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 msgr2=0x55f1b7e49050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e49050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c010 tx=0x7f6af4009260 comp rx=0 tx=0).stop 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 shutdown_connections 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7e27240 0x55f1b7e29660 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e49050 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/192104089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e49590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.305+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 >> 172.21.15.64:0/192104089 conn(0x55f1b7d841b0 msgr2=0x55f1b7e20860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.306+0000 7f6ae2ffd700 0 client.25402 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.306+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 shutdown_connections 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.306+0000 7f6b161a9b80 1 -- 172.21.15.64:0/192104089 wait complete. 2024-05-08T22:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.306+0000 7f6b161a9b80 20 client.25402 trim_cache size 0 max 16384 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 Processor -- start 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 -- start start 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e50590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7e47bb0 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7e4a2a0 con 0x55f1b7e2fae0 2024-05-08T22:01:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.325+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7e23b60 con 0x55f1b7e2e290 2024-05-08T22:01:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56050/0 (socket says 172.21.15.64:56050) 2024-05-08T22:01:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e50590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b037fe700 1 -- 172.21.15.64:0/613754542 learned_addr learned my addr 172.21.15.64:0/613754542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 -- 172.21.15.64:0/613754542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 msgr2=0x55f1b7e2cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 -- 172.21.15.64:0/613754542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2fae0 msgr2=0x55f1b7e35f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 -- 172.21.15.64:0/613754542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1b7d3c500 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.326+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e50590 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6af80ca170 tx=0x7f6af80ca070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17b45bd02553696 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.327+0000 7f6aea7fc700 1 -- 172.21.15.64:0/613754542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80a1f70 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.327+0000 7f6aea7fc700 1 -- 172.21.15.64:0/613754542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af80ca2b0 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.327+0000 7f6aea7fc700 1 -- 172.21.15.64:0/613754542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8099fb0 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 -- 172.21.15.64:0/613754542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 msgr2=0x55f1b7e50590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e50590 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6af80ca170 tx=0x7f6af80ca070 comp rx=0 tx=0).stop 2024-05-08T22:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 -- 172.21.15.64:0/613754542 shutdown_connections 2024-05-08T22:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e35f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2e290 0x55f1b7e2cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/613754542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e50590 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 -- 172.21.15.64:0/613754542 >> 172.21.15.64:0/613754542 conn(0x55f1b7d841b0 msgr2=0x55f1b7e48560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 -- 172.21.15.64:0/613754542 shutdown_connections 2024-05-08T22:01:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 -- 172.21.15.64:0/613754542 wait complete. 2024-05-08T22:01:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.328+0000 7f6b161a9b80 1 -- start start 2024-05-08T22:01:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 Processor -- start 2024-05-08T22:01:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 0x55f1b7e4c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e4ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e4fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7e500f0 con 0x55f1b7e2e290 2024-05-08T22:01:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7e50260 con 0x55f1b7e2ebf0 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b7e50420 con 0x55f1b7e2fae0 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b161a9b80 10 client.0 _ll_register_callbacks cb 0x55f1b7d1bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 0x55f1b7e4c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 0x55f1b7e4c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36688/0 (socket says 172.21.15.64:36688) 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 -- 172.21.15.64:0/4129488912 learned_addr learned my addr 172.21.15.64:0/4129488912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e4ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b03fff700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e4fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 -- 172.21.15.64:0/4129488912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 msgr2=0x55f1b7e4fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e4fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 -- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 msgr2=0x55f1b7e4ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e4ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b037fe700 1 -- 172.21.15.64:0/4129488912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b041a1630 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.329+0000 7f6b03fff700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e4fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6b037fe700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 0x55f1b7e4c790 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000c00 tx=0x7f6af4017a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d2af7a48b47d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6ae2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/4129488912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4017ea0 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/4129488912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6af4003680 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6afc00f040 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/4129488912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af402e390 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6b161a9b80 10 client.15765 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.330+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f1b7bfbc70 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/4129488912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc00f040 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/4129488912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6af4037050 con 0x55f1b7e2e290 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b161a9b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b161a9b80 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b161a9b80 20 client.15765 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7eb4110 0x55f1b7eb6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b161a9b80 4 client.15765 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f1b7d6ee60 con 0x55f1b7eb4110 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.331+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7eb4110 0x55f1b7eb6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.332+0000 7f6ae2ffd700 10 client.15765 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.332+0000 7f6b02ffd700 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7eb4110 0x55f1b7eb6500 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69f07470ab52e38 server_cookie=c8e3ee0d34b6d54f in_seq=0 out_seq=0 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6ae2ffd700 1 -- 172.21.15.64:0/4129488912 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f1b7d6ee60 con 0x55f1b7eb4110 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6ae2ffd700 10 client.15765 handle_command_reply: tid=0 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 client.15765 shutdown 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 msgr2=0x55f1b7e4c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 0x55f1b7e4c790 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000c00 tx=0x7f6af4017a80 comp rx=0 tx=0).stop 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 shutdown_connections 2024-05-08T22:01:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55f1b7eb4110 0x55f1b7eb6500 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1b7e2fae0 0x55f1b7e4fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1b7e2e290 0x55f1b7e4c790 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 --2- 172.21.15.64:0/4129488912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f1b7e2ebf0 0x55f1b7e4ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 >> 172.21.15.64:0/4129488912 conn(0x55f1b7d841b0 msgr2=0x55f1b7e48440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6ae2ffd700 0 client.15765 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 shutdown_connections 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 1 -- 172.21.15.64:0/4129488912 wait complete. 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.342+0000 7f6b161a9b80 20 client.15765 trim_cache size 0 max 16384 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6aec0a1170 msgr2=0x7f6aec0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6aec0a1170 0x7f6aec0a3630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005950 tx=0x7f6af4019040 comp rx=0 tx=0).stop 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 msgr2=0x7f6b04199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 0x7f6b04199710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6af800d950 tx=0x7f6af800dc60 comp rx=0 tx=0).stop 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 shutdown_connections 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6aec0a1170 0x7f6aec0a3630 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b04101990 0x7f6b041a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b040ffe00 0x7f6b041991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 --2- 172.21.15.64:0/765821095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b04100ff0 0x7f6b04199710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 >> 172.21.15.64:0/765821095 conn(0x7f6b040fb390 msgr2=0x7f6b040682f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 shutdown_connections 2024-05-08T22:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:29.343+0000 7f6b09fef700 1 -- 172.21.15.64:0/765821095 wait complete. 2024-05-08T22:01:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:30 smithi186 ceph-mon[46619]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 924 op/s 2024-05-08T22:01:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:30 smithi064 ceph-mon[42500]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 924 op/s 2024-05-08T22:01:30.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:30 smithi160 ceph-mon[46962]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 924 op/s 2024-05-08T22:01:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 400.0 s, 1509.4 tps, lat 66.176 ms stddev 108.621 2024-05-08T22:01:32.401 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:32 smithi186 ceph-mon[46619]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.52k op/s 2024-05-08T22:01:32.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:32.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:32.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:32.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:01:32.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:32 smithi064 ceph-mon[42500]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.52k op/s 2024-05-08T22:01:32.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:32.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:32.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:32.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:01:32.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:32 smithi160 ceph-mon[46962]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.52k op/s 2024-05-08T22:01:32.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:32.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:32.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:32.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:01:34.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:34 smithi186 ceph-mon[46619]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2024-05-08T22:01:34.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:34 smithi064 ceph-mon[42500]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2024-05-08T22:01:34.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:34 smithi160 ceph-mon[46962]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:36 smithi186 ceph-mon[46619]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 1.91k op/s 2024-05-08T22:01:36.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:36 smithi064 ceph-mon[42500]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 1.91k op/s 2024-05-08T22:01:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 405.0 s, 1842.6 tps, lat 54.226 ms stddev 64.565 2024-05-08T22:01:36.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:36 smithi160 ceph-mon[46962]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 1.91k op/s 2024-05-08T22:01:38.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:38 smithi186 ceph-mon[46619]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2024-05-08T22:01:38.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:38 smithi064 ceph-mon[42500]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2024-05-08T22:01:38.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:38 smithi160 ceph-mon[46962]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2024-05-08T22:01:40.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:40 smithi186 ceph-mon[46619]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 MiB/s wr, 1.85k op/s 2024-05-08T22:01:40.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:40 smithi064 ceph-mon[42500]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 MiB/s wr, 1.85k op/s 2024-05-08T22:01:40.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:40 smithi160 ceph-mon[46962]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 MiB/s wr, 1.85k op/s 2024-05-08T22:01:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 410.0 s, 2114.2 tps, lat 47.859 ms stddev 58.106 2024-05-08T22:01:42.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:42 smithi064 ceph-mon[42500]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.79k op/s 2024-05-08T22:01:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.528+0000 7fb72f874700 1 -- 172.21.15.64:0/3232473241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 msgr2=0x7fb72810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.528+0000 7fb72f874700 1 --2- 172.21.15.64:0/3232473241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 0x7fb72810cda0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009990 tx=0x7fb71c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.529+0000 7fb72f874700 1 -- 172.21.15.64:0/3232473241 shutdown_connections 2024-05-08T22:01:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.529+0000 7fb72f874700 1 --2- 172.21.15.64:0/3232473241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72810d2e0 0x7fb72810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.529+0000 7fb72f874700 1 --2- 172.21.15.64:0/3232473241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb728107fa0 0x7fb72810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.529+0000 7fb72f874700 1 --2- 172.21.15.64:0/3232473241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 0x7fb72810cda0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.529+0000 7fb72f874700 1 -- 172.21.15.64:0/3232473241 >> 172.21.15.64:0/3232473241 conn(0x7fb72806df30 msgr2=0x7fb728070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.531+0000 7fb72f874700 1 -- 172.21.15.64:0/3232473241 shutdown_connections 2024-05-08T22:01:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.531+0000 7fb72f874700 1 -- 172.21.15.64:0/3232473241 wait complete. 2024-05-08T22:01:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 Processor -- start 2024-05-08T22:01:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 -- start start 2024-05-08T22:01:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb728107fa0 0x7fb7281a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 0x7fb7281a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 0x7fb7281ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281ad860 con 0x7fb728107fa0 2024-05-08T22:01:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281ad9d0 con 0x7fb72810a910 2024-05-08T22:01:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.532+0000 7fb72f874700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281adb90 con 0x7fb72810d2e0 2024-05-08T22:01:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.533+0000 7fb72de11700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 0x7fb7281ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.533+0000 7fb72de11700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 0x7fb7281ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:49586/0 (socket says 172.21.15.64:49586) 2024-05-08T22:01:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.533+0000 7fb72de11700 1 -- 172.21.15.64:0/3142608319 learned_addr learned my addr 172.21.15.64:0/3142608319 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.533+0000 7fb72d610700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb728107fa0 0x7fb7281a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72de11700 1 -- 172.21.15.64:0/3142608319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 msgr2=0x7fb7281a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72de11700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 0x7fb7281a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72de11700 1 -- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb728107fa0 msgr2=0x7fb7281a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72de11700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb728107fa0 0x7fb7281a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72de11700 1 -- 172.21.15.64:0/3142608319 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0095f0 con 0x7fb72810d2e0 2024-05-08T22:01:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72de11700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 0x7fb7281ad290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d980 tx=0x7fb72000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb71a7fc700 1 -- 172.21.15.64:0/3142608319 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000bb60 con 0x7fb72810d2e0 2024-05-08T22:01:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7281ade70 con 0x7fb72810d2e0 2024-05-08T22:01:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.534+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7281ae2c0 con 0x7fb72810d2e0 2024-05-08T22:01:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.536+0000 7fb71a7fc700 1 -- 172.21.15.64:0/3142608319 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7200075e0 con 0x7fb72810d2e0 2024-05-08T22:01:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.536+0000 7fb71a7fc700 1 -- 172.21.15.64:0/3142608319 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720004e10 con 0x7fb72810d2e0 2024-05-08T22:01:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.539+0000 7fb73ba2eb80 1 Processor -- start 2024-05-08T22:01:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.540+0000 7fb71a7fc700 1 -- 172.21.15.64:0/3142608319 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb7200050a0 con 0x7fb72810d2e0 2024-05-08T22:01:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.541+0000 7fb71a7fc700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7140a1220 0x7fb7140a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.541+0000 7fb71a7fc700 1 -- 172.21.15.64:0/3142608319 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7200cc720 con 0x7fb72810d2e0 2024-05-08T22:01:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.541+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7140a1220 0x7fb7140a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.541+0000 7fb73ba2eb80 1 -- start start 2024-05-08T22:01:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e23750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7140a1220 0x7fb7140a36e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0095c0 tx=0x7fb71c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949d790f0 con 0x55b949e223e0 2024-05-08T22:01:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949d76ab0 con 0x55b949e232d0 2024-05-08T22:01:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949d723a0 con 0x55b949e21a80 2024-05-08T22:01:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e22cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43426/0 (socket says 172.21.15.64:43426) 2024-05-08T22:01:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 -- 172.21.15.64:0/1848919720 learned_addr learned my addr 172.21.15.64:0/1848919720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72de11700 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e23750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72d610700 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 -- 172.21.15.64:0/1848919720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 msgr2=0x55b949e21ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e21ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 -- 172.21.15.64:0/1848919720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 msgr2=0x55b949e23750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e23750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 -- 172.21.15.64:0/1848919720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b949bf3350 con 0x55b949e223e0 2024-05-08T22:01:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.542+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e22cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb71c015000 tx=0x7fb71c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0dd8cf4ff2fe706 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:42.549 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:42 smithi160 ceph-mon[46962]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.79k op/s 2024-05-08T22:01:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.545+0000 7fb7137fe700 1 -- 172.21.15.64:0/1848919720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c01d070 con 0x55b949e223e0 2024-05-08T22:01:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.545+0000 7fb7137fe700 1 -- 172.21.15.64:0/1848919720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71c00bab0 con 0x55b949e223e0 2024-05-08T22:01:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.545+0000 7fb7137fe700 1 -- 172.21.15.64:0/1848919720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00f900 con 0x55b949e223e0 2024-05-08T22:01:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/1848919720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 msgr2=0x55b949e22cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e22cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb71c015000 tx=0x7fb71c01d000 comp rx=0 tx=0).stop 2024-05-08T22:01:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/1848919720 shutdown_connections 2024-05-08T22:01:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e23750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e21ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/1848919720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e22cf0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/1848919720 >> 172.21.15.64:0/1848919720 conn(0x55b949d801b0 msgr2=0x55b949e1e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/1848919720 shutdown_connections 2024-05-08T22:01:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.546+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/1848919720 wait complete. 2024-05-08T22:01:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 -- start start 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 Processor -- start 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e2b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e223e0 0x55b949e2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e232d0 0x55b949e32e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949d0e500 con 0x55b949e232d0 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949d0e1a0 con 0x55b949e223e0 2024-05-08T22:01:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949bf5640 con 0x55b949e21a80 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.547+0000 7fb73ba2eb80 10 client.0 _ll_register_callbacks cb 0x55b949d17ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72de11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e232d0 0x55b949e32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72de11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e232d0 0x55b949e32e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43432/0 (socket says 172.21.15.64:43432) 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72de11700 1 -- 172.21.15.64:0/2646000450 learned_addr learned my addr 172.21.15.64:0/2646000450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e223e0 0x55b949e2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2646000450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 msgr2=0x55b949e2b880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:01:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e2b880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2646000450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e232d0 msgr2=0x55b949e32e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e232d0 0x55b949e32e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.548+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2646000450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72810e250 con 0x55b949e223e0 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e223e0 0x55b949e2bdc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb71c000c00 tx=0x7fb71c0053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5f08eef50b92e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb710ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb710ff9700 1 -- 172.21.15.64:0/2646000450 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00fa90 con 0x55b949e223e0 2024-05-08T22:01:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb710ff9700 1 -- 172.21.15.64:0/2646000450 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb71c005650 con 0x55b949e223e0 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b949ccb210 con 0x55b949e223e0 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb710ff9700 1 -- 172.21.15.64:0/2646000450 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb71c00fbf0 con 0x55b949e223e0 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb73ba2eb80 10 client.25511 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.549+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b949b9fb00 con 0x55b949e223e0 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb710ff9700 1 -- 172.21.15.64:0/2646000450 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b949ccb210 con 0x55b949e223e0 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb710ff9700 1 -- 172.21.15.64:0/2646000450 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb71c024420 con 0x55b949e223e0 2024-05-08T22:01:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb73ba2eb80 10 client.25511 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb73ba2eb80 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb73ba2eb80 20 client.25511 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e48170 0x55b949e4a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb73ba2eb80 4 client.25511 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b949df17b0 con 0x55b949e48170 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb72d610700 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e48170 0x55b949e4a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.550+0000 7fb710ff9700 10 client.25511 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb72d610700 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e48170 0x55b949e4a590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792019df70869d98 server_cookie=f766e0d9a8d05ced in_seq=0 out_seq=0 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb710ff9700 1 -- 172.21.15.64:0/2646000450 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b949df17b0 con 0x55b949e48170 2024-05-08T22:01:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb710ff9700 10 client.25511 handle_command_reply: tid=0 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 client.25511 shutdown 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e223e0 msgr2=0x55b949e2bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e223e0 0x55b949e2bdc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb71c000c00 tx=0x7fb71c0053b0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 shutdown_connections 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e48170 0x55b949e4a590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e232d0 0x55b949e32e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb710ff9700 0 client.25511 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e21a80 0x55b949e2b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2646000450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e223e0 0x55b949e2bdc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.551+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 >> 172.21.15.64:0/2646000450 conn(0x55b949d801b0 msgr2=0x55b949e1c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.552+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 shutdown_connections 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.552+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2646000450 wait complete. 2024-05-08T22:01:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.552+0000 7fb73ba2eb80 20 client.25511 trim_cache size 0 max 16384 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 Processor -- start 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 -- start start 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 0x55b949e4a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e223e0 0x55b949e32dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949e199b0 con 0x55b949e21a80 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949e2a330 con 0x55b949e232d0 2024-05-08T22:01:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.572+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949e47fe0 con 0x55b949e223e0 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 0x55b949e4a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 0x55b949e4a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43440/0 (socket says 172.21.15.64:43440) 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 -- 172.21.15.64:0/2231755779 learned_addr learned my addr 172.21.15.64:0/2231755779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 -- 172.21.15.64:0/2231755779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e223e0 msgr2=0x55b949e32dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e223e0 0x55b949e32dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 -- 172.21.15.64:0/2231755779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 msgr2=0x55b949e236b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 -- 172.21.15.64:0/2231755779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b949d38500 con 0x55b949e21a80 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb72d610700 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 0x55b949e4a590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55b949e4ae70 tx=0x7fb7240094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f02ebb955b31ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb713fff700 1 -- 172.21.15.64:0/2231755779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400ac60 con 0x55b949e21a80 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb713fff700 1 -- 172.21.15.64:0/2231755779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb72401b070 con 0x55b949e21a80 2024-05-08T22:01:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.573+0000 7fb713fff700 1 -- 172.21.15.64:0/2231755779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240208c0 con 0x55b949e21a80 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2231755779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 msgr2=0x55b949e4a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 0x55b949e4a590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55b949e4ae70 tx=0x7fb7240094f0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2231755779 shutdown_connections 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e232d0 0x55b949e236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e21a80 0x55b949e4a590 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2231755779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e223e0 0x55b949e32dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2231755779 >> 172.21.15.64:0/2231755779 conn(0x55b949d801b0 msgr2=0x55b949e298d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2231755779 shutdown_connections 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2231755779 wait complete. 2024-05-08T22:01:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 -- start start 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.574+0000 7fb73ba2eb80 1 Processor -- start 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e21a80 0x55b949e302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e232d0 0x55b949e317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949e31ce0 con 0x55b949e223e0 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949e31e50 con 0x55b949e21a80 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949e32010 con 0x55b949e232d0 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb73ba2eb80 10 client.0 _ll_register_callbacks cb 0x55b949d17ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e30830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43454/0 (socket says 172.21.15.64:43454) 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2400548578 learned_addr learned my addr 172.21.15.64:0/2400548578 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72d610700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e21a80 0x55b949e302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72de11700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e232d0 0x55b949e317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2400548578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e232d0 msgr2=0x55b949e317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e232d0 0x55b949e317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e21a80 msgr2=0x55b949e302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e21a80 0x55b949e302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72ce0f700 1 -- 172.21.15.64:0/2400548578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb724009990 con 0x55b949e223e0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.575+0000 7fb72d610700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e21a80 0x55b949e302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.576+0000 7fb72ce0f700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e30830 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009990 tx=0x7fb71c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173285bf8f0c014d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.576+0000 7fb710ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.576+0000 7fb710ff9700 1 -- 172.21.15.64:0/2400548578 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c01d040 con 0x55b949e223e0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.576+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb71c00bab0 con 0x55b949e223e0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.576+0000 7fb710ff9700 1 -- 172.21.15.64:0/2400548578 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb71c03d020 con 0x55b949e223e0 2024-05-08T22:01:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.576+0000 7fb710ff9700 1 -- 172.21.15.64:0/2400548578 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb71c00f4d0 con 0x55b949e223e0 2024-05-08T22:01:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.578+0000 7fb73ba2eb80 10 client.15783 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.578+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b949bf7c70 con 0x55b949e223e0 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.578+0000 7fb710ff9700 1 -- 172.21.15.64:0/2400548578 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71c00bab0 con 0x55b949e223e0 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.578+0000 7fb710ff9700 1 -- 172.21.15.64:0/2400548578 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb71c04a070 con 0x55b949e223e0 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb73ba2eb80 10 client.15783 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb73ba2eb80 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb73ba2eb80 20 client.15783 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e9e870 0x55b949ea0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb73ba2eb80 4 client.15783 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b949d6ae60 con 0x55b949e9e870 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb72d610700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e9e870 0x55b949ea0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.579+0000 7fb710ff9700 10 client.15783 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.580+0000 7fb72d610700 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e9e870 0x55b949ea0c60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c033376bc4530f9 server_cookie=1fff7346f7d509ff in_seq=0 out_seq=0 2024-05-08T22:01:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.591+0000 7fb710ff9700 1 -- 172.21.15.64:0/2400548578 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b949d6ae60 con 0x55b949e9e870 2024-05-08T22:01:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.591+0000 7fb710ff9700 10 client.15783 handle_command_reply: tid=0 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 client.15783 shutdown 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 msgr2=0x55b949e30830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e30830 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009990 tx=0x7fb71c009310 comp rx=0 tx=0).stop 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 shutdown_connections 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b949e21a80 0x55b949e302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b949e9e870 0x55b949ea0c60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b949e232d0 0x55b949e317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 --2- 172.21.15.64:0/2400548578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b949e223e0 0x55b949e30830 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 >> 172.21.15.64:0/2400548578 conn(0x55b949d801b0 msgr2=0x55b949e1e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb710ff9700 0 client.15783 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 shutdown_connections 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 1 -- 172.21.15.64:0/2400548578 wait complete. 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.592+0000 7fb73ba2eb80 20 client.15783 trim_cache size 0 max 16384 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:01:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7140a1220 msgr2=0x7fb7140a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7140a1220 0x7fb7140a36e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0095c0 tx=0x7fb71c009550 comp rx=0 tx=0).stop 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 msgr2=0x7fb7281ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 0x7fb7281ad290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d980 tx=0x7fb72000dd40 comp rx=0 tx=0).stop 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 shutdown_connections 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb7140a1220 0x7fb7140a36e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb72810d2e0 0x7fb7281ad290 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb728107fa0 0x7fb7281a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 --2- 172.21.15.64:0/3142608319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb72810a910 0x7fb7281a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 >> 172.21.15.64:0/3142608319 conn(0x7fb72806df30 msgr2=0x7fb7281083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 shutdown_connections 2024-05-08T22:01:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:42.593+0000 7fb72f874700 1 -- 172.21.15.64:0/3142608319 wait complete. 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:42 smithi186 ceph-mon[46619]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.79k op/s 2024-05-08T22:01:44.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:44 smithi064 ceph-mon[42500]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.17k op/s 2024-05-08T22:01:44.549 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:44 smithi160 ceph-mon[46962]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.17k op/s 2024-05-08T22:01:44.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:44 smithi186 ceph-mon[46619]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.17k op/s 2024-05-08T22:01:46.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:46 smithi064 ceph-mon[42500]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 787 op/s 2024-05-08T22:01:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 415.0 s, 2146.0 tps, lat 46.157 ms stddev 57.424 2024-05-08T22:01:46.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:46 smithi160 ceph-mon[46962]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 787 op/s 2024-05-08T22:01:46.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:46 smithi186 ceph-mon[46619]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 787 op/s 2024-05-08T22:01:48.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:48 smithi064 ceph-mon[42500]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 700 op/s 2024-05-08T22:01:48.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:48 smithi160 ceph-mon[46962]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 700 op/s 2024-05-08T22:01:48.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:48 smithi186 ceph-mon[46619]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 700 op/s 2024-05-08T22:01:50.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:50 smithi064 ceph-mon[42500]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 680 op/s 2024-05-08T22:01:50.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:50 smithi160 ceph-mon[46962]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 680 op/s 2024-05-08T22:01:50.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:50 smithi186 ceph-mon[46619]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 680 op/s 2024-05-08T22:01:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 420.0 s, 1838.6 tps, lat 54.620 ms stddev 62.562 2024-05-08T22:01:52.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:52 smithi064 ceph-mon[42500]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-08T22:01:52.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:52 smithi160 ceph-mon[46962]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-08T22:01:52.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:52 smithi186 ceph-mon[46619]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-08T22:01:54.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:54 smithi160 ceph-mon[46962]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 676 op/s 2024-05-08T22:01:54.648 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:54 smithi186 ceph-mon[46619]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 676 op/s 2024-05-08T22:01:54.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:54 smithi064 ceph-mon[42500]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 676 op/s 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 -- 172.21.15.64:0/2572190327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6240ff6a0 msgr2=0x7fb624101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/2572190327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6240ff6a0 0x7fb624101ad0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb620009990 tx=0x7fb620009ca0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 -- 172.21.15.64:0/2572190327 shutdown_connections 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/2572190327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6241049e0 0x7fb624104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/2572190327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6240ff6a0 0x7fb624101ad0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/2572190327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb624102010 0x7fb6241044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 -- 172.21.15.64:0/2572190327 >> 172.21.15.64:0/2572190327 conn(0x7fb6240fb280 msgr2=0x7fb6240fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 -- 172.21.15.64:0/2572190327 shutdown_connections 2024-05-08T22:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.753+0000 7fb62bfaa700 1 -- 172.21.15.64:0/2572190327 wait complete. 2024-05-08T22:01:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 Processor -- start 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 -- start start 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6240ff6a0 0x7fb62410fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 0x7fb62410e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6241049e0 0x7fb62410e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624110350 con 0x7fb624102010 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62410ebd0 con 0x7fb6241049e0 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62bfaa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62410ed90 con 0x7fb6240ff6a0 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb629545700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 0x7fb62410e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb629d46700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6240ff6a0 0x7fb62410fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb629545700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 0x7fb62410e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38734/0 (socket says 172.21.15.64:38734) 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb629545700 1 -- 172.21.15.64:0/896554575 learned_addr learned my addr 172.21.15.64:0/896554575 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.754+0000 7fb62a547700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6241049e0 0x7fb62410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb629545700 1 -- 172.21.15.64:0/896554575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6240ff6a0 msgr2=0x7fb62410fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb629545700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6240ff6a0 0x7fb62410fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb629545700 1 -- 172.21.15.64:0/896554575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6241049e0 msgr2=0x7fb62410e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb629545700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6241049e0 0x7fb62410e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb629545700 1 -- 172.21.15.64:0/896554575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6200095f0 con 0x7fb624102010 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb629545700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 0x7fb62410e150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb61400e9a0 tx=0x7fb61400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb61affd700 1 -- 172.21.15.64:0/896554575 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009790 con 0x7fb624102010 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb61affd700 1 -- 172.21.15.64:0/896554575 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb614010430 con 0x7fb624102010 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6241add90 con 0x7fb624102010 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb61affd700 1 -- 172.21.15.64:0/896554575 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614018680 con 0x7fb624102010 2024-05-08T22:01:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.755+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6241ae1b0 con 0x7fb624102010 2024-05-08T22:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.760+0000 7fb638164b80 1 Processor -- start 2024-05-08T22:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.760+0000 7fb61affd700 1 -- 172.21.15.64:0/896554575 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb6140105a0 con 0x7fb624102010 2024-05-08T22:01:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.761+0000 7fb61affd700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6100a11d0 0x7fb6100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.761+0000 7fb61affd700 1 -- 172.21.15.64:0/896554575 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb614014070 con 0x7fb624102010 2024-05-08T22:01:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.761+0000 7fb62a547700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6100a11d0 0x7fb6100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.761+0000 7fb638164b80 1 -- start start 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb62a547700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6100a11d0 0x7fb6100a3690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00f540 tx=0x7fb61c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb638164b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff2751ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb638164b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff2752cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb638164b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2753750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb638164b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff26a90f0 con 0x556ff2751a80 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb638164b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff26a6ab0 con 0x556ff27523e0 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb638164b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff26a23a0 con 0x556ff27532d0 2024-05-08T22:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff2751ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb62a547700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2753750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff2751ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38746/0 (socket says 172.21.15.64:38746) 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 -- 172.21.15.64:0/4165564682 learned_addr learned my addr 172.21.15.64:0/4165564682 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb62a547700 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2753750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:35416/0 (socket says 172.21.15.64:35416) 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629545700 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff2752cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 -- 172.21.15.64:0/4165564682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 msgr2=0x556ff2753750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2753750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 -- 172.21.15.64:0/4165564682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 msgr2=0x556ff2752cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff2752cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.762+0000 7fb629d46700 1 -- 172.21.15.64:0/4165564682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ff2523350 con 0x556ff2751a80 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.763+0000 7fb629545700 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff2752cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.763+0000 7fb629d46700 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff2751ea0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb620015000 tx=0x7fb620005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f027e9e5dba431 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.763+0000 7fb60ffff700 1 -- 172.21.15.64:0/4165564682 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62001d070 con 0x556ff2751a80 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.763+0000 7fb60ffff700 1 -- 172.21.15.64:0/4165564682 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb62001e040 con 0x556ff2751a80 2024-05-08T22:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.763+0000 7fb60ffff700 1 -- 172.21.15.64:0/4165564682 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620003680 con 0x556ff2751a80 2024-05-08T22:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 -- 172.21.15.64:0/4165564682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 msgr2=0x556ff2751ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff2751ea0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb620015000 tx=0x7fb620005a90 comp rx=0 tx=0).stop 2024-05-08T22:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 -- 172.21.15.64:0/4165564682 shutdown_connections 2024-05-08T22:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2753750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff2751ea0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 --2- 172.21.15.64:0/4165564682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff2752cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.764+0000 7fb638164b80 1 -- 172.21.15.64:0/4165564682 >> 172.21.15.64:0/4165564682 conn(0x556ff26b01b0 msgr2=0x556ff274e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.765+0000 7fb638164b80 1 -- 172.21.15.64:0/4165564682 shutdown_connections 2024-05-08T22:01:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.765+0000 7fb638164b80 1 -- 172.21.15.64:0/4165564682 wait complete. 2024-05-08T22:01:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.765+0000 7fb638164b80 1 -- start start 2024-05-08T22:01:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.765+0000 7fb638164b80 1 Processor -- start 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff275b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff275bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2762dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff263e500 con 0x556ff2751a80 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff263e1a0 con 0x556ff27523e0 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff2525640 con 0x556ff27532d0 2024-05-08T22:01:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb638164b80 10 client.0 _ll_register_callbacks cb 0x556ff2647ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb629d46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff275b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb629d46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff275b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38748/0 (socket says 172.21.15.64:38748) 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.766+0000 7fb629d46700 1 -- 172.21.15.64:0/1335774673 learned_addr learned my addr 172.21.15.64:0/1335774673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb629d46700 1 -- 172.21.15.64:0/1335774673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 msgr2=0x556ff2762dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb629d46700 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2762dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb629d46700 1 -- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 msgr2=0x556ff275bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:01:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb629d46700 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff275bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb629d46700 1 -- 172.21.15.64:0/1335774673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb624105990 con 0x556ff2751a80 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb629d46700 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff275b840 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb620009ce0 tx=0x7fb620005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee9ee61649d9db4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb60d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb60d7fa700 1 -- 172.21.15.64:0/1335774673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62001d070 con 0x556ff2751a80 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb60d7fa700 1 -- 172.21.15.64:0/1335774673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb620003880 con 0x556ff2751a80 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.767+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ff25fb210 con 0x556ff2751a80 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb60d7fa700 1 -- 172.21.15.64:0/1335774673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb620005da0 con 0x556ff2751a80 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 10 client.15795 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x556ff24cfb00 con 0x556ff2751a80 2024-05-08T22:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb60d7fa700 1 -- 172.21.15.64:0/1335774673 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ff25fb210 con 0x556ff2751a80 2024-05-08T22:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb60d7fa700 1 -- 172.21.15.64:0/1335774673 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb620004c20 con 0x556ff2751a80 2024-05-08T22:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 20 client.15795 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff2767240 0x556ff2769660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 4 client.15795 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.768+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ff27217b0 con 0x556ff2767240 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.769+0000 7fb629545700 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff2767240 0x556ff2769660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.769+0000 7fb629545700 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff2767240 0x556ff2769660 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffdd05860ff5bcd server_cookie=5146c27b33fa8285 in_seq=0 out_seq=0 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.769+0000 7fb60d7fa700 10 client.15795 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb60d7fa700 1 -- 172.21.15.64:0/1335774673 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ff27217b0 con 0x556ff2767240 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb60d7fa700 10 client.15795 handle_command_reply: tid=0 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 client.15795 shutdown 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 msgr2=0x556ff275b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff275b840 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb620009ce0 tx=0x7fb620005780 comp rx=0 tx=0).stop 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 shutdown_connections 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff2767240 0x556ff2769660 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27532d0 0x556ff2762dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff275b840 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 --2- 172.21.15.64:0/1335774673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27523e0 0x556ff275bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 >> 172.21.15.64:0/1335774673 conn(0x556ff26b01b0 msgr2=0x556ff274c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb60d7fa700 0 client.15795 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 shutdown_connections 2024-05-08T22:01:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.770+0000 7fb638164b80 1 -- 172.21.15.64:0/1335774673 wait complete. 2024-05-08T22:01:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.771+0000 7fb638164b80 20 client.15795 trim_cache size 0 max 16384 2024-05-08T22:01:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.795+0000 7fb638164b80 1 Processor -- start 2024-05-08T22:01:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.795+0000 7fb638164b80 1 -- start start 2024-05-08T22:01:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.795+0000 7fb638164b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff27536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.795+0000 7fb638164b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27523e0 0x556ff2762ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.795+0000 7fb638164b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff2756510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.795+0000 7fb638164b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff2763220 con 0x556ff2751a80 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb638164b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff2751630 con 0x556ff27532d0 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb638164b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff275a0c0 con 0x556ff27523e0 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff2756510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff2756510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:41644/0 (socket says 172.21.15.64:41644) 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 -- 172.21.15.64:0/1323015137 learned_addr learned my addr 172.21.15.64:0/1323015137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 -- 172.21.15.64:0/1323015137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27523e0 msgr2=0x556ff2762ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27523e0 0x556ff2762ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 -- 172.21.15.64:0/1323015137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 msgr2=0x556ff27536b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff27536b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.796+0000 7fb62a547700 1 -- 172.21.15.64:0/1323015137 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ff2668500 con 0x556ff27532d0 2024-05-08T22:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.803+0000 7fb62a547700 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff2756510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009390 tx=0x7fb61c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30e870d11516534 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.815+0000 7fb618ff9700 1 -- 172.21.15.64:0/1323015137 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c01c040 con 0x556ff27532d0 2024-05-08T22:01:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.815+0000 7fb618ff9700 1 -- 172.21.15.64:0/1323015137 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61c00f040 con 0x556ff27532d0 2024-05-08T22:01:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.815+0000 7fb618ff9700 1 -- 172.21.15.64:0/1323015137 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e040 con 0x556ff27532d0 2024-05-08T22:01:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.816+0000 7fb638164b80 1 -- 172.21.15.64:0/1323015137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 msgr2=0x556ff2756510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.816+0000 7fb638164b80 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff2756510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009390 tx=0x7fb61c014000 comp rx=0 tx=0).stop 2024-05-08T22:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.820+0000 7fb638164b80 1 -- 172.21.15.64:0/1323015137 shutdown_connections 2024-05-08T22:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.820+0000 7fb638164b80 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff2756510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.820+0000 7fb638164b80 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff2751a80 0x556ff27536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.820+0000 7fb638164b80 1 --2- 172.21.15.64:0/1323015137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff27523e0 0x556ff2762ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.820+0000 7fb638164b80 1 -- 172.21.15.64:0/1323015137 >> 172.21.15.64:0/1323015137 conn(0x556ff26b01b0 msgr2=0x556ff275ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.828+0000 7fb638164b80 1 -- 172.21.15.64:0/1323015137 shutdown_connections 2024-05-08T22:01:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.830+0000 7fb638164b80 1 -- 172.21.15.64:0/1323015137 wait complete. 2024-05-08T22:01:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 -- start start 2024-05-08T22:01:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 Processor -- start 2024-05-08T22:01:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 0x556ff275ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff27523e0 0x556ff27604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff27bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff2760ac0 con 0x556ff27523e0 2024-05-08T22:01:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff27bbc20 con 0x556ff27532d0 2024-05-08T22:01:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff27ccd70 con 0x556ff2751a80 2024-05-08T22:01:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.831+0000 7fb638164b80 10 client.0 _ll_register_callbacks cb 0x556ff2647ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:01:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.833+0000 7fb629d46700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 0x556ff275ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.833+0000 7fb629d46700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 0x556ff275ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:35444/0 (socket says 172.21.15.64:35444) 2024-05-08T22:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.833+0000 7fb629d46700 1 -- 172.21.15.64:0/704716286 learned_addr learned my addr 172.21.15.64:0/704716286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.833+0000 7fb629545700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff27523e0 0x556ff27604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.833+0000 7fb62a547700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff27bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb629d46700 1 -- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 msgr2=0x556ff27bb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb629d46700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff27bb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb629d46700 1 -- 172.21.15.64:0/704716286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff27523e0 msgr2=0x556ff27604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:01:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb629d46700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff27523e0 0x556ff27604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb629d46700 1 -- 172.21.15.64:0/704716286 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c009040 con 0x556ff2751a80 2024-05-08T22:01:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb629d46700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 0x556ff275ff60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb620000c00 tx=0x7fb620001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e263c79a834482 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.834+0000 7fb60d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:01:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.836+0000 7fb60d7fa700 1 -- 172.21.15.64:0/704716286 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62001d070 con 0x556ff2751a80 2024-05-08T22:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.836+0000 7fb60d7fa700 1 -- 172.21.15.64:0/704716286 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb62000f600 con 0x556ff2751a80 2024-05-08T22:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.836+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb62001e040 con 0x556ff2751a80 2024-05-08T22:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.836+0000 7fb60d7fa700 1 -- 172.21.15.64:0/704716286 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb62000b2e0 con 0x556ff2751a80 2024-05-08T22:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.836+0000 7fb638164b80 10 client.25423 fetch_fsmap learned FSMap version 39 2024-05-08T22:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.836+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x556ff2527c70 con 0x556ff2751a80 2024-05-08T22:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb60d7fa700 1 -- 172.21.15.64:0/704716286 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb62001e040 con 0x556ff2751a80 2024-05-08T22:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb60d7fa700 1 -- 172.21.15.64:0/704716286 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb620003680 con 0x556ff2751a80 2024-05-08T22:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb638164b80 10 client.25423 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb638164b80 10 client.25423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb638164b80 20 client.25423 populate_metadata read hostname 'smithi064' 2024-05-08T22:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb638164b80 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff27ce250 0x556ff27d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb638164b80 4 client.25423 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.841+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ff269ae60 con 0x556ff27ce250 2024-05-08T22:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.846+0000 7fb629545700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff27ce250 0x556ff27d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.846+0000 7fb60d7fa700 10 client.25423 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.846+0000 7fb629545700 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff27ce250 0x556ff27d0670 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fc1074a349ab82 server_cookie=b43047634a7f5348 in_seq=0 out_seq=0 2024-05-08T22:01:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.860+0000 7fb60d7fa700 1 -- 172.21.15.64:0/704716286 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ff269ae60 con 0x556ff27ce250 2024-05-08T22:01:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.860+0000 7fb60d7fa700 10 client.25423 handle_command_reply: tid=0 2024-05-08T22:01:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.860+0000 7fb638164b80 1 client.25423 shutdown 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 msgr2=0x556ff275ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 0x556ff275ff60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb620000c00 tx=0x7fb620001e50 comp rx=0 tx=0).stop 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 shutdown_connections 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ff27532d0 0x556ff27bb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556ff27ce250 0x556ff27d0670 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ff2751a80 0x556ff275ff60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 --2- 172.21.15.64:0/704716286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ff27523e0 0x556ff27604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 >> 172.21.15.64:0/704716286 conn(0x556ff26b01b0 msgr2=0x556ff2759760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb60d7fa700 0 client.25423 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 shutdown_connections 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.861+0000 7fb638164b80 1 -- 172.21.15.64:0/704716286 wait complete. 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb638164b80 20 client.25423 trim_cache size 0 max 16384 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:01:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6100a11d0 msgr2=0x7fb6100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6100a11d0 0x7fb6100a3690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00f540 tx=0x7fb61c00b040 comp rx=0 tx=0).stop 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 msgr2=0x7fb62410e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 0x7fb62410e150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb61400e9a0 tx=0x7fb61400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 shutdown_connections 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb6100a11d0 0x7fb6100a3690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6241049e0 0x7fb62410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6240ff6a0 0x7fb62410fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 --2- 172.21.15.64:0/896554575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb624102010 0x7fb62410e150 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.862+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 >> 172.21.15.64:0/896554575 conn(0x7fb6240fb280 msgr2=0x7fb6240fb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.863+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 shutdown_connections 2024-05-08T22:01:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:01:55.867+0000 7fb62bfaa700 1 -- 172.21.15.64:0/896554575 wait complete. 2024-05-08T22:01:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 425.0 s, 1786.0 tps, lat 55.309 ms stddev 63.822 2024-05-08T22:01:56.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:56 smithi160 ceph-mon[46962]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 701 op/s 2024-05-08T22:01:56.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:56 smithi186 ceph-mon[46619]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 701 op/s 2024-05-08T22:01:56.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:56 smithi064 ceph-mon[42500]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 701 op/s 2024-05-08T22:01:58.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:58 smithi160 ceph-mon[46962]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 688 op/s 2024-05-08T22:01:58.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:58 smithi186 ceph-mon[46619]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 688 op/s 2024-05-08T22:01:58.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:58 smithi064 ceph-mon[42500]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 688 op/s 2024-05-08T22:01:59.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:01:59 smithi186 ceph-mon[46619]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 703 op/s 2024-05-08T22:01:59.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:01:59 smithi064 ceph-mon[42500]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 703 op/s 2024-05-08T22:01:59.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:01:59 smithi160 ceph-mon[46962]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 703 op/s 2024-05-08T22:02:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 430.0 s, 1356.7 tps, lat 74.024 ms stddev 87.742 2024-05-08T22:02:01.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:01 smithi186 ceph-mon[46619]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 818 op/s 2024-05-08T22:02:01.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:01 smithi064 ceph-mon[42500]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 818 op/s 2024-05-08T22:02:01.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:01 smithi160 ceph-mon[46962]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 818 op/s 2024-05-08T22:02:03.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:03 smithi186 ceph-mon[46619]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 803 op/s 2024-05-08T22:02:03.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:03 smithi064 ceph-mon[42500]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 803 op/s 2024-05-08T22:02:03.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:03 smithi160 ceph-mon[46962]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 803 op/s 2024-05-08T22:02:05.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:05 smithi064 ceph-mon[42500]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 820 op/s 2024-05-08T22:02:05.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:05 smithi160 ceph-mon[46962]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 820 op/s 2024-05-08T22:02:05.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:05 smithi186 ceph-mon[46619]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 820 op/s 2024-05-08T22:02:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 435.0 s, 1554.4 tps, lat 64.540 ms stddev 72.430 2024-05-08T22:02:07.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:07 smithi064 ceph-mon[42500]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 860 op/s 2024-05-08T22:02:07.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:07 smithi160 ceph-mon[46962]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 860 op/s 2024-05-08T22:02:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:07 smithi186 ceph-mon[46619]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 860 op/s 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.066+0000 7f242cd70700 1 -- 172.21.15.64:0/2608125711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 msgr2=0x7f2428109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.066+0000 7f242cd70700 1 --2- 172.21.15.64:0/2608125711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428109500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f242000b0a0 tx=0x7f242000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 -- 172.21.15.64:0/2608125711 shutdown_connections 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 --2- 172.21.15.64:0/2608125711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428109500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 --2- 172.21.15.64:0/2608125711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2428102520 0x7f2428102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 --2- 172.21.15.64:0/2608125711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24280fef90 0x7f24280ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 -- 172.21.15.64:0/2608125711 >> 172.21.15.64:0/2608125711 conn(0x7f24280faaf0 msgr2=0x7f24280fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 -- 172.21.15.64:0/2608125711 shutdown_connections 2024-05-08T22:02:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.067+0000 7f242cd70700 1 -- 172.21.15.64:0/2608125711 wait complete. 2024-05-08T22:02:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 Processor -- start 2024-05-08T22:02:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 -- start start 2024-05-08T22:02:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24280fef90 0x7f2428194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2428102520 0x7f242819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242819c830 con 0x7f24280fef90 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242819c9a0 con 0x7f24280ff930 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.068+0000 7f242cd70700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242819cb60 con 0x7f2428102520 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:55978/0 (socket says 172.21.15.64:55978) 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 -- 172.21.15.64:0/1303158804 learned_addr learned my addr 172.21.15.64:0/1303158804 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2426d9d700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2428102520 0x7f242819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 -- 172.21.15.64:0/1303158804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2428102520 msgr2=0x7f242819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2428102520 0x7f242819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 -- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24280fef90 msgr2=0x7f2428194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24280fef90 0x7f2428194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 -- 172.21.15.64:0/1303158804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420009d00 con 0x7f24280ff930 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f2425d9b700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428195280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f241c00d950 tx=0x7f241c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f24177fe700 1 -- 172.21.15.64:0/1303158804 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c009b40 con 0x7f24280ff930 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f24177fe700 1 -- 172.21.15.64:0/1303158804 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241c009ca0 con 0x7f24280ff930 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f24177fe700 1 -- 172.21.15.64:0/1303158804 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00f620 con 0x7f24280ff930 2024-05-08T22:02:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.069+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242819ce40 con 0x7f24280ff930 2024-05-08T22:02:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.070+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242819d2c0 con 0x7f24280ff930 2024-05-08T22:02:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.075+0000 7f2438f2ab80 1 Processor -- start 2024-05-08T22:02:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.075+0000 7f24177fe700 1 -- 172.21.15.64:0/1303158804 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f241c010460 con 0x7f24280ff930 2024-05-08T22:02:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.076+0000 7f24177fe700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f24100a11a0 0x7f24100a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.076+0000 7f24177fe700 1 -- 172.21.15.64:0/1303158804 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f241c0ccbe0 con 0x7f24280ff930 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.078+0000 7f2426d9d700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f24100a11a0 0x7f24100a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.079+0000 7f2426d9d700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f24100a11a0 0x7f24100a3660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f242000b780 tx=0x7f24200060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 -- start start 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c50cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c51750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8ba70f0 con 0x55b3a8c503e0 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8ba4ab0 con 0x55b3a8c512d0 2024-05-08T22:02:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.080+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8ba03a0 con 0x55b3a8c4fa80 2024-05-08T22:02:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c51750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c51750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:55982/0 (socket says 172.21.15.64:55982) 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 -- 172.21.15.64:0/965097024 learned_addr learned my addr 172.21.15.64:0/965097024 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f242659c700 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 -- 172.21.15.64:0/965097024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c4fa80 msgr2=0x55b3a8c4fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 -- 172.21.15.64:0/965097024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 msgr2=0x55b3a8c50cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c50cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 -- 172.21.15.64:0/965097024 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3a8a21350 con 0x55b3a8c512d0 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2426d9d700 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c51750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f242000b0a0 tx=0x7f242001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd209a9a5ac4b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2414ff9700 1 -- 172.21.15.64:0/965097024 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000e040 con 0x55b3a8c512d0 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2414ff9700 1 -- 172.21.15.64:0/965097024 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f242001d450 con 0x55b3a8c512d0 2024-05-08T22:02:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.081+0000 7f2414ff9700 1 -- 172.21.15.64:0/965097024 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200052a0 con 0x55b3a8c512d0 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 -- 172.21.15.64:0/965097024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 msgr2=0x55b3a8c51750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c51750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f242000b0a0 tx=0x7f242001a000 comp rx=0 tx=0).stop 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 -- 172.21.15.64:0/965097024 shutdown_connections 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c51750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/965097024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c50cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 -- 172.21.15.64:0/965097024 >> 172.21.15.64:0/965097024 conn(0x55b3a8bae1b0 msgr2=0x55b3a8c4c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.082+0000 7f2438f2ab80 1 -- 172.21.15.64:0/965097024 shutdown_connections 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 -- 172.21.15.64:0/965097024 wait complete. 2024-05-08T22:02:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 -- start start 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 Processor -- start 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8b3c500 con 0x55b3a8c503e0 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8b3c1a0 con 0x55b3a8c4fa80 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8a23640 con 0x55b3a8c512d0 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.083+0000 7f2438f2ab80 10 client.0 _ll_register_callbacks cb 0x55b3a8b45ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58050/0 (socket says 172.21.15.64:58050) 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 -- 172.21.15.64:0/28738471 learned_addr learned my addr 172.21.15.64:0/28738471 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f242659c700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2426d9d700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 -- 172.21.15.64:0/28738471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 msgr2=0x55b3a8c69630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 -- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 msgr2=0x55b3a8c620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 -- 172.21.15.64:0/28738471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428109cc0 con 0x55b3a8c503e0 2024-05-08T22:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2426d9d700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.084+0000 7f2425d9b700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c625f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f241c099d40 tx=0x7f241c099a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393fd0a034352deb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f240dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f240dffb700 1 -- 172.21.15.64:0/28738471 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c010910 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f240dffb700 1 -- 172.21.15.64:0/28738471 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f241c09de70 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3a8af9210 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f240dffb700 1 -- 172.21.15.64:0/28738471 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241c0a24c0 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f2438f2ab80 10 client.15804 fetch_fsmap learned FSMap version 39 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b3a89cdb00 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f240dffb700 1 -- 172.21.15.64:0/28738471 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3a8af9210 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.086+0000 7f240dffb700 1 -- 172.21.15.64:0/28738471 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f241c099530 con 0x55b3a8c503e0 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f2438f2ab80 10 client.15804 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:02:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f2438f2ab80 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f2438f2ab80 20 client.15804 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8c6df00 0x55b3a8c70320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f2438f2ab80 4 client.15804 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:02:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3a8c1f7b0 con 0x55b3a8c6df00 2024-05-08T22:02:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.087+0000 7f242659c700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8c6df00 0x55b3a8c70320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.088+0000 7f242659c700 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8c6df00 0x55b3a8c70320 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923fa690d8a6861 server_cookie=5009569fea0842f2 in_seq=0 out_seq=0 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.088+0000 7f240dffb700 10 client.15804 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.088+0000 7f240dffb700 1 -- 172.21.15.64:0/28738471 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3a8c1f7b0 con 0x55b3a8c6df00 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.088+0000 7f240dffb700 10 client.15804 handle_command_reply: tid=0 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.088+0000 7f2438f2ab80 1 client.15804 shutdown 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.089+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 msgr2=0x55b3a8c625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.089+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c625f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f241c099d40 tx=0x7f241c099a90 comp rx=0 tx=0).stop 2024-05-08T22:02:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 shutdown_connections 2024-05-08T22:02:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8c6df00 0x55b3a8c70320 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/28738471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c625f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 >> 172.21.15.64:0/28738471 conn(0x55b3a8bae1b0 msgr2=0x55b3a8c4a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.092+0000 7f240dffb700 0 client.15804 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.093+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 shutdown_connections 2024-05-08T22:02:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.093+0000 7f2438f2ab80 1 -- 172.21.15.64:0/28738471 wait complete. 2024-05-08T22:02:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.093+0000 7f2438f2ab80 20 client.15804 trim_cache size 0 max 16384 2024-05-08T22:02:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.115+0000 7f2438f2ab80 1 Processor -- start 2024-05-08T22:02:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.115+0000 7f2438f2ab80 1 -- start start 2024-05-08T22:02:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8c51150 con 0x55b3a8c503e0 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8c62fb0 con 0x55b3a8c4fa80 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8c45780 con 0x55b3a8c512d0 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:55996/0 (socket says 172.21.15.64:55996) 2024-05-08T22:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 -- 172.21.15.64:0/4045305780 learned_addr learned my addr 172.21.15.64:0/4045305780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2425d9b700 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f2426d9d700 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 -- 172.21.15.64:0/4045305780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 msgr2=0x55b3a8c69ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 -- 172.21.15.64:0/4045305780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 msgr2=0x55b3a8c695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.116+0000 7f242659c700 1 -- 172.21.15.64:0/4045305780 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3a8b66500 con 0x55b3a8c4fa80 2024-05-08T22:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.120+0000 7f242659c700 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fe60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24281961c0 tx=0x7f241801a9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d2d57eca3370b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.120+0000 7f24157fa700 1 -- 172.21.15.64:0/4045305780 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418012040 con 0x55b3a8c4fa80 2024-05-08T22:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.120+0000 7f24157fa700 1 -- 172.21.15.64:0/4045305780 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2418013040 con 0x55b3a8c4fa80 2024-05-08T22:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.120+0000 7f24157fa700 1 -- 172.21.15.64:0/4045305780 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418025b80 con 0x55b3a8c4fa80 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 -- 172.21.15.64:0/4045305780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 msgr2=0x55b3a8c4fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fe60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24281961c0 tx=0x7f241801a9b0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 -- 172.21.15.64:0/4045305780 shutdown_connections 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c4fe60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c512d0 0x55b3a8c69ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/4045305780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c503e0 0x55b3a8c695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 -- 172.21.15.64:0/4045305780 >> 172.21.15.64:0/4045305780 conn(0x55b3a8bae1b0 msgr2=0x55b3a8c578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 -- 172.21.15.64:0/4045305780 shutdown_connections 2024-05-08T22:02:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.121+0000 7f2438f2ab80 1 -- 172.21.15.64:0/4045305780 wait complete. 2024-05-08T22:02:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 -- start start 2024-05-08T22:02:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 Processor -- start 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 0x55b3a8c654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c68330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8c68870 con 0x55b3a8c4fa80 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8c689e0 con 0x55b3a8c512d0 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a8c68b50 con 0x55b3a8c503e0 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2438f2ab80 10 client.0 _ll_register_callbacks cb 0x55b3a8b45ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 0x55b3a8c654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.122+0000 7f242659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 0x55b3a8c654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:46632/0 (socket says 172.21.15.64:46632) 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 -- 172.21.15.64:0/2163620346 learned_addr learned my addr 172.21.15.64:0/2163620346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2426d9d700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c68330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 -- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 msgr2=0x55b3a8c68330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c68330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 -- 172.21.15.64:0/2163620346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c4fa80 msgr2=0x55b3a8c64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 -- 172.21.15.64:0/2163620346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241801a400 con 0x55b3a8c503e0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f242659c700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2426d9d700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c68330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2425d9b700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 0x55b3a8c654a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f241c09a8f0 tx=0x7f241c096380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd68ba1f57b0048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f240dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f240dffb700 1 -- 172.21.15.64:0/2163620346 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0a20d0 con 0x55b3a8c503e0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f240dffb700 1 -- 172.21.15.64:0/2163620346 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f241c0a2290 con 0x55b3a8c503e0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f242001d450 con 0x55b3a8c503e0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.123+0000 7f240dffb700 1 -- 172.21.15.64:0/2163620346 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241c0cadb0 con 0x55b3a8c503e0 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 10 client.25435 fetch_fsmap learned FSMap version 39 2024-05-08T22:02:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b3a8a25c70 con 0x55b3a8c503e0 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f240dffb700 1 -- 172.21.15.64:0/2163620346 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f242001d450 con 0x55b3a8c503e0 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f240dffb700 1 -- 172.21.15.64:0/2163620346 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f241c09a2a0 con 0x55b3a8c503e0 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 10 client.25435 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 10 client.25435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 20 client.25435 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8cd58e0 0x55b3a8cd7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 4 client.25435 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b3a8b98e60 con 0x55b3a8cd58e0 2024-05-08T22:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.124+0000 7f242659c700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8cd58e0 0x55b3a8cd7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.125+0000 7f240dffb700 10 client.25435 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.125+0000 7f242659c700 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8cd58e0 0x55b3a8cd7d00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bdf20579c279c6 server_cookie=86721b9b411f664 in_seq=0 out_seq=0 2024-05-08T22:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.133+0000 7f240dffb700 1 -- 172.21.15.64:0/2163620346 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b3a8b98e60 con 0x55b3a8cd58e0 2024-05-08T22:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.133+0000 7f240dffb700 10 client.25435 handle_command_reply: tid=0 2024-05-08T22:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.133+0000 7f2438f2ab80 1 client.25435 shutdown 2024-05-08T22:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 msgr2=0x55b3a8c654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 0x55b3a8c654a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f241c09a8f0 tx=0x7f241c096380 comp rx=0 tx=0).stop 2024-05-08T22:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 shutdown_connections 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3a8c512d0 0x55b3a8c68330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b3a8cd58e0 0x55b3a8cd7d00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3a8c4fa80 0x55b3a8c64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 --2- 172.21.15.64:0/2163620346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a8c503e0 0x55b3a8c654a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f240dffb700 0 client.25435 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 >> 172.21.15.64:0/2163620346 conn(0x55b3a8bae1b0 msgr2=0x55b3a8c57880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 shutdown_connections 2024-05-08T22:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 1 -- 172.21.15.64:0/2163620346 wait complete. 2024-05-08T22:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.134+0000 7f2438f2ab80 20 client.25435 trim_cache size 0 max 16384 2024-05-08T22:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f24100a11a0 msgr2=0x7f24100a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f24100a11a0 0x7f24100a3660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f242000b780 tx=0x7f24200060c0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 msgr2=0x7f2428195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428195280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f241c00d950 tx=0x7f241c00dd10 comp rx=0 tx=0).stop 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 shutdown_connections 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f24100a11a0 0x7f24100a3660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2428102520 0x7f242819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24280fef90 0x7f2428194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 --2- 172.21.15.64:0/1303158804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ff930 0x7f2428195280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 >> 172.21.15.64:0/1303158804 conn(0x7f24280faaf0 msgr2=0x7f24281055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 shutdown_connections 2024-05-08T22:02:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:09.135+0000 7f242cd70700 1 -- 172.21.15.64:0/1303158804 wait complete. 2024-05-08T22:02:09.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:09 smithi064 ceph-mon[42500]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 861 op/s 2024-05-08T22:02:09.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:09 smithi160 ceph-mon[46962]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 861 op/s 2024-05-08T22:02:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:09 smithi186 ceph-mon[46619]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 861 op/s 2024-05-08T22:02:11.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 440.0 s, 1379.8 tps, lat 71.603 ms stddev 82.278 2024-05-08T22:02:11.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:11 smithi064 ceph-mon[42500]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 895 op/s 2024-05-08T22:02:11.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:11 smithi160 ceph-mon[46962]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 895 op/s 2024-05-08T22:02:11.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:11 smithi186 ceph-mon[46619]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 895 op/s 2024-05-08T22:02:13.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:13 smithi160 ceph-mon[46962]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 779 op/s 2024-05-08T22:02:13.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:13 smithi186 ceph-mon[46619]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 779 op/s 2024-05-08T22:02:13.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:13 smithi064 ceph-mon[42500]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 779 op/s 2024-05-08T22:02:15.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:15 smithi160 ceph-mon[46962]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 MiB/s wr, 1.29k op/s 2024-05-08T22:02:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:15 smithi186 ceph-mon[46619]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 MiB/s wr, 1.29k op/s 2024-05-08T22:02:15.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:15 smithi064 ceph-mon[42500]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 MiB/s wr, 1.29k op/s 2024-05-08T22:02:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 445.0 s, 1214.8 tps, lat 83.330 ms stddev 137.496 2024-05-08T22:02:17.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:17 smithi160 ceph-mon[46962]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2024-05-08T22:02:17.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:17 smithi186 ceph-mon[46619]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2024-05-08T22:02:17.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:17 smithi064 ceph-mon[42500]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2024-05-08T22:02:19.797 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:19 smithi160 ceph-mon[46962]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.94k op/s 2024-05-08T22:02:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:19 smithi186 ceph-mon[46619]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.94k op/s 2024-05-08T22:02:19.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:19 smithi064 ceph-mon[42500]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.94k op/s 2024-05-08T22:02:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 450.0 s, 1710.9 tps, lat 58.009 ms stddev 62.706 2024-05-08T22:02:21.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:21 smithi160 ceph-mon[46962]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2024-05-08T22:02:21.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:21 smithi186 ceph-mon[46619]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2024-05-08T22:02:21.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:21 smithi064 ceph-mon[42500]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 -- 172.21.15.64:0/249617277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70100f20 msgr2=0x7fed70101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 --2- 172.21.15.64:0/249617277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70100f20 0x7fed70101340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed600098f0 tx=0x7fed60009c00 comp rx=0 tx=0).stop 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 -- 172.21.15.64:0/249617277 shutdown_connections 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 --2- 172.21.15.64:0/249617277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70100f20 0x7fed70101340 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 --2- 172.21.15.64:0/249617277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed70102a30 0x7fed70109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 --2- 172.21.15.64:0/249617277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102070 0x7fed701024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 -- 172.21.15.64:0/249617277 >> 172.21.15.64:0/249617277 conn(0x7fed700fc490 msgr2=0x7fed700fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 -- 172.21.15.64:0/249617277 shutdown_connections 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 -- 172.21.15.64:0/249617277 wait complete. 2024-05-08T22:02:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.296+0000 7fed7595f700 1 Processor -- start 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 -- start start 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed70100f20 0x7fed701a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70102070 0x7fed701a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 0x7fed701a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701a3be0 con 0x7fed70100f20 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701a3d50 con 0x7fed70102070 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.297+0000 7fed7595f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701a3f10 con 0x7fed70102a30 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 0x7fed701a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 0x7fed701a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:54790/0 (socket says 172.21.15.64:54790) 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 -- 172.21.15.64:0/552819276 learned_addr learned my addr 172.21.15.64:0/552819276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 -- 172.21.15.64:0/552819276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70102070 msgr2=0x7fed701a6df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70102070 0x7fed701a6df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 -- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed70100f20 msgr2=0x7fed701a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed70100f20 0x7fed701a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.298+0000 7fed6f7fe700 1 -- 172.21.15.64:0/552819276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640095f0 con 0x7fed70102a30 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.299+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 0x7fed701a3440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c370 tx=0x7fed6800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.299+0000 7fed7495d700 1 -- 172.21.15.64:0/552819276 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68017070 con 0x7fed70102a30 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.299+0000 7fed7495d700 1 -- 172.21.15.64:0/552819276 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6800f040 con 0x7fed70102a30 2024-05-08T22:02:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.299+0000 7fed7495d700 1 -- 172.21.15.64:0/552819276 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800e050 con 0x7fed70102a30 2024-05-08T22:02:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.300+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed68009c40 con 0x7fed70102a30 2024-05-08T22:02:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.300+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed60009520 con 0x7fed70102a30 2024-05-08T22:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.303+0000 7fed7495d700 1 -- 172.21.15.64:0/552819276 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fed68004540 con 0x7fed70102a30 2024-05-08T22:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 Processor -- start 2024-05-08T22:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 -- start start 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a5a80 0x55e3116a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a72d0 0x55e3116a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3115fd0f0 con 0x55e3116a72d0 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3115faab0 con 0x55e3116a63e0 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.305+0000 7fed81b19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3115f63a0 con 0x55e3116a5a80 2024-05-08T22:02:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a5a80 0x55e3116a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45116/0 (socket says 172.21.15.64:45116) 2024-05-08T22:02:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 -- 172.21.15.64:0/3364883746 learned_addr learned my addr 172.21.15.64:0/3364883746 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6e7fc700 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a72d0 0x55e3116a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 -- 172.21.15.64:0/3364883746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a5a80 msgr2=0x55e3116a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a5a80 0x55e3116a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 -- 172.21.15.64:0/3364883746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a72d0 msgr2=0x55e3116a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a72d0 0x55e3116a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6effd700 1 -- 172.21.15.64:0/3364883746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e311477350 con 0x55e3116a63e0 2024-05-08T22:02:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.306+0000 7fed6e7fc700 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a5a80 0x55e3116a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed6effd700 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116a6cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed60009810 tx=0x7fed600058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea07e5b1b0fd79e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed55ffb700 1 -- 172.21.15.64:0/3364883746 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6001c070 con 0x55e3116a63e0 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed55ffb700 1 -- 172.21.15.64:0/3364883746 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6000fbb0 con 0x55e3116a63e0 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed55ffb700 1 -- 172.21.15.64:0/3364883746 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60005040 con 0x55e3116a63e0 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed7495d700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fed580a11a0 0x7fed580a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed7495d700 1 -- 172.21.15.64:0/552819276 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed680cbda0 con 0x7fed70102a30 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed6e7fc700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fed580a11a0 0x7fed580a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed81b19b80 1 -- 172.21.15.64:0/3364883746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 msgr2=0x55e3116a6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.307+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116a6cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed60009810 tx=0x7fed600058e0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 -- 172.21.15.64:0/3364883746 shutdown_connections 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a72d0 0x55e3116a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a5a80 0x55e3116a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3364883746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116a6cf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 -- 172.21.15.64:0/3364883746 >> 172.21.15.64:0/3364883746 conn(0x55e3116041b0 msgr2=0x55e3116a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 -- 172.21.15.64:0/3364883746 shutdown_connections 2024-05-08T22:02:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 -- 172.21.15.64:0/3364883746 wait complete. 2024-05-08T22:02:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.308+0000 7fed81b19b80 1 -- start start 2024-05-08T22:02:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed6e7fc700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fed580a11a0 0x7fed580a3660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55e3116a6ab0 tx=0x7fed64009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 Processor -- start 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a5a80 0x55e3116af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a63e0 0x55e3116afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311592500 con 0x55e3116a63e0 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3115921a0 con 0x55e3116a5a80 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311479640 con 0x55e3116a72d0 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed81b19b80 10 client.0 _ll_register_callbacks cb 0x55e31159bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:54810/0 (socket says 172.21.15.64:54810) 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed6f7fe700 1 -- 172.21.15.64:0/3722666780 learned_addr learned my addr 172.21.15.64:0/3722666780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.309+0000 7fed6effd700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a5a80 0x55e3116af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6f7fe700 1 -- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a5a80 msgr2=0x55e3116af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a5a80 0x55e3116af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6f7fe700 1 -- 172.21.15.64:0/3722666780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a63e0 msgr2=0x55e3116afd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a63e0 0x55e3116afd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6f7fe700 1 -- 172.21.15.64:0/3722666780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed70109c80 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6effd700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a5a80 0x55e3116af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b6db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed68095c70 tx=0x7fed68099200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18be85f5259fb480 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed4effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed4effd700 1 -- 172.21.15.64:0/3722666780 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed680af070 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed4effd700 1 -- 172.21.15.64:0/3722666780 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed680a1680 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e31154f210 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed4effd700 1 -- 172.21.15.64:0/3722666780 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed680975a0 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed81b19b80 10 client.25447 fetch_fsmap learned FSMap version 39 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.310+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e311423b00 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed4effd700 1 -- 172.21.15.64:0/3722666780 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e31154f210 con 0x55e3116a72d0 2024-05-08T22:02:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed4effd700 1 -- 172.21.15.64:0/3722666780 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed6809c020 con 0x55e3116a72d0 2024-05-08T22:02:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed81b19b80 10 client.25447 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:02:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed81b19b80 10 client.25447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed81b19b80 20 client.25447 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e3116bb5e0 0x55e3116bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed81b19b80 4 client.25447 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:02:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3116757b0 con 0x55e3116bb5e0 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.311+0000 7fed6effd700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e3116bb5e0 0x55e3116bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.312+0000 7fed6effd700 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e3116bb5e0 0x55e3116bda00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97615d64952830e server_cookie=4c63f6be3c29e1e0 in_seq=0 out_seq=0 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.312+0000 7fed4effd700 10 client.25447 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.312+0000 7fed4effd700 1 -- 172.21.15.64:0/3722666780 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3116757b0 con 0x55e3116bb5e0 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.312+0000 7fed4effd700 10 client.25447 handle_command_reply: tid=0 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.312+0000 7fed81b19b80 1 client.25447 shutdown 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 msgr2=0x55e3116b6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b6db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed68095c70 tx=0x7fed68099200 comp rx=0 tx=0).stop 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 shutdown_connections 2024-05-08T22:02:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a5a80 0x55e3116af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e3116bb5e0 0x55e3116bda00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b6db0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 --2- 172.21.15.64:0/3722666780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a63e0 0x55e3116afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 >> 172.21.15.64:0/3722666780 conn(0x55e3116041b0 msgr2=0x55e3116a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed4effd700 0 client.25447 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 shutdown_connections 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.313+0000 7fed81b19b80 1 -- 172.21.15.64:0/3722666780 wait complete. 2024-05-08T22:02:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.314+0000 7fed81b19b80 20 client.25447 trim_cache size 0 max 16384 2024-05-08T22:02:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.332+0000 7fed81b19b80 1 Processor -- start 2024-05-08T22:02:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.332+0000 7fed81b19b80 1 -- start start 2024-05-08T22:02:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a63e0 0x55e3116b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a72d0 0x55e3116a2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed81b19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31169b330 con 0x55e3116a5a80 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed81b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3116a5520 con 0x55e3116a72d0 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed81b19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3116b8b40 con 0x55e3116a63e0 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a72d0 0x55e3116a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35418/0 (socket says 172.21.15.64:35418) 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6effd700 1 -- 172.21.15.64:0/1512312935 learned_addr learned my addr 172.21.15.64:0/1512312935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6e7fc700 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a63e0 0x55e3116b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6f7fe700 1 -- 172.21.15.64:0/1512312935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a63e0 msgr2=0x55e3116b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a63e0 0x55e3116b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6f7fe700 1 -- 172.21.15.64:0/1512312935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 msgr2=0x55e3116a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6f7fe700 1 -- 172.21.15.64:0/1512312935 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3115bc500 con 0x55e3116a72d0 2024-05-08T22:02:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.333+0000 7fed6effd700 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.334+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a72d0 0x55e3116a2020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed68099840 tx=0x7fed6809d190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241b24b653a266f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.334+0000 7fed567fc700 1 -- 172.21.15.64:0/1512312935 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed680af070 con 0x55e3116a72d0 2024-05-08T22:02:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.334+0000 7fed567fc700 1 -- 172.21.15.64:0/1512312935 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed680981b0 con 0x55e3116a72d0 2024-05-08T22:02:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.334+0000 7fed567fc700 1 -- 172.21.15.64:0/1512312935 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68098950 con 0x55e3116a72d0 2024-05-08T22:02:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 -- 172.21.15.64:0/1512312935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a72d0 msgr2=0x55e3116a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a72d0 0x55e3116a2020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed68099840 tx=0x7fed6809d190 comp rx=0 tx=0).stop 2024-05-08T22:02:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 -- 172.21.15.64:0/1512312935 shutdown_connections 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a72d0 0x55e3116a2020 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 --2- 172.21.15.64:0/1512312935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a63e0 0x55e3116b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 -- 172.21.15.64:0/1512312935 >> 172.21.15.64:0/1512312935 conn(0x55e3116041b0 msgr2=0x55e3116c23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 -- 172.21.15.64:0/1512312935 shutdown_connections 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 -- 172.21.15.64:0/1512312935 wait complete. 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.335+0000 7fed81b19b80 1 -- start start 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 Processor -- start 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3116b5c80 con 0x55e3116a5a80 2024-05-08T22:02:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3116b5df0 con 0x55e3116a63e0 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3116b5fb0 con 0x55e3116a72d0 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed81b19b80 10 client.0 _ll_register_callbacks cb 0x55e31159bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:54826/0 (socket says 172.21.15.64:54826) 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed6f7fe700 1 -- 172.21.15.64:0/523074145 learned_addr learned my addr 172.21.15.64:0/523074145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.336+0000 7fed6e7fc700 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed6f7fe700 1 -- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 msgr2=0x55e3116b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed6f7fe700 1 -- 172.21.15.64:0/523074145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 msgr2=0x55e3116b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed6f7fe700 1 -- 172.21.15.64:0/523074145 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e311477350 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed6f7fe700 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b5740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed6800cff0 tx=0x7fed68095670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316eb49cf06d4263 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed4effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed4effd700 1 -- 172.21.15.64:0/523074145 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed680af030 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed4effd700 1 -- 172.21.15.64:0/523074145 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed680b0070 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.337+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6000fbb0 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed4effd700 1 -- 172.21.15.64:0/523074145 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed680a1680 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 10 client.25453 fetch_fsmap learned FSMap version 39 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e311423b00 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed4effd700 1 -- 172.21.15.64:0/523074145 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6000fbb0 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed4effd700 1 -- 172.21.15.64:0/523074145 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed6809c070 con 0x55e3116a72d0 2024-05-08T22:02:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 10 client.25453 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 20 client.25453 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e311711d90 0x55e311714180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 4 client.25453 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3115eee60 con 0x55e311711d90 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.338+0000 7fed6effd700 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e311711d90 0x55e311714180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.340+0000 7fed6effd700 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e311711d90 0x55e311714180 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a0787d120b7714 server_cookie=877dbfde838aa857 in_seq=0 out_seq=0 2024-05-08T22:02:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.340+0000 7fed4effd700 10 client.25453 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.349+0000 7fed4effd700 1 -- 172.21.15.64:0/523074145 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3115eee60 con 0x55e311711d90 2024-05-08T22:02:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.349+0000 7fed4effd700 10 client.25453 handle_command_reply: tid=0 2024-05-08T22:02:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.349+0000 7fed81b19b80 1 client.25453 shutdown 2024-05-08T22:02:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 msgr2=0x55e3116b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b5740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed6800cff0 tx=0x7fed68095670 comp rx=0 tx=0).stop 2024-05-08T22:02:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 shutdown_connections 2024-05-08T22:02:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e311711d90 0x55e311714180 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3116a72d0 0x55e3116b5740 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3116a5a80 0x55e3116b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 --2- 172.21.15.64:0/523074145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e3116a63e0 0x55e3116b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 >> 172.21.15.64:0/523074145 conn(0x55e3116041b0 msgr2=0x55e3116ad400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.351+0000 7fed4effd700 0 client.25453 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.352+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 shutdown_connections 2024-05-08T22:02:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.352+0000 7fed81b19b80 1 -- 172.21.15.64:0/523074145 wait complete. 2024-05-08T22:02:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.352+0000 7fed81b19b80 20 client.25453 trim_cache size 0 max 16384 2024-05-08T22:02:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:02:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:02:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:02:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:02:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fed580a11a0 msgr2=0x7fed580a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fed580a11a0 0x7fed580a3660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55e3116a6ab0 tx=0x7fed64009310 comp rx=0 tx=0).stop 2024-05-08T22:02:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 msgr2=0x7fed701a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 0x7fed701a3440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c370 tx=0x7fed6800c730 comp rx=0 tx=0).stop 2024-05-08T22:02:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 shutdown_connections 2024-05-08T22:02:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fed580a11a0 0x7fed580a3660 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70102a30 0x7fed701a3440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed70100f20 0x7fed701a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 --2- 172.21.15.64:0/552819276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed70102070 0x7fed701a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 >> 172.21.15.64:0/552819276 conn(0x7fed700fc490 msgr2=0x7fed701054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 shutdown_connections 2024-05-08T22:02:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:22.353+0000 7fed7595f700 1 -- 172.21.15.64:0/552819276 wait complete. 2024-05-08T22:02:23.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:23 smithi160 ceph-mon[46962]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2024-05-08T22:02:23.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:23 smithi186 ceph-mon[46619]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2024-05-08T22:02:23.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:23 smithi064 ceph-mon[42500]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2024-05-08T22:02:25.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:25 smithi160 ceph-mon[46962]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 1.88k op/s 2024-05-08T22:02:25.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:25 smithi186 ceph-mon[46619]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 1.88k op/s 2024-05-08T22:02:25.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:25 smithi064 ceph-mon[42500]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 1.88k op/s 2024-05-08T22:02:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 455.0 s, 1863.0 tps, lat 54.321 ms stddev 62.060 2024-05-08T22:02:27.899 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:27 smithi186 ceph-mon[46619]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-05-08T22:02:27.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:27 smithi064 ceph-mon[42500]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-05-08T22:02:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:27 smithi160 ceph-mon[46962]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-05-08T22:02:29.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:29 smithi186 ceph-mon[46619]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 857 op/s 2024-05-08T22:02:29.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:29 smithi064 ceph-mon[42500]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 857 op/s 2024-05-08T22:02:30.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:29 smithi160 ceph-mon[46962]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 857 op/s 2024-05-08T22:02:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 460.0 s, 2168.0 tps, lat 46.352 ms stddev 53.080 2024-05-08T22:02:31.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:31 smithi186 ceph-mon[46619]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 778 op/s 2024-05-08T22:02:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:31 smithi064 ceph-mon[42500]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 778 op/s 2024-05-08T22:02:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:31 smithi160 ceph-mon[46962]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 778 op/s 2024-05-08T22:02:32.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:32 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:32 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:33.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:32 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 709 op/s 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: Unable to set osd_memory_target on smithi160 to 191373414: error parsing value: Value '191373414' is below minimum 939524096 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi064 to 182.5M 2024-05-08T22:02:33.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: Unable to set osd_memory_target on smithi064 to 191382732: error parsing value: Value '191382732' is below minimum 939524096 2024-05-08T22:02:33.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:33.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:33.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:02:33.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 709 op/s 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: Unable to set osd_memory_target on smithi160 to 191373414: error parsing value: Value '191373414' is below minimum 939524096 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi064 to 182.5M 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: Unable to set osd_memory_target on smithi064 to 191382732: error parsing value: Value '191382732' is below minimum 939524096 2024-05-08T22:02:33.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:33.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:33.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 709 op/s 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: Unable to set osd_memory_target on smithi160 to 191373414: error parsing value: Value '191373414' is below minimum 939524096 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:34.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi064 to 182.5M 2024-05-08T22:02:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: Unable to set osd_memory_target on smithi064 to 191382732: error parsing value: Value '191382732' is below minimum 939524096 2024-05-08T22:02:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:34.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:02:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.510+0000 7f72595b7700 1 -- 172.21.15.64:0/3205758722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7254106c20 msgr2=0x7f7254109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.510+0000 7f72595b7700 1 --2- 172.21.15.64:0/3205758722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7254106c20 0x7f7254109090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f724c00b0a0 tx=0x7f724c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.511+0000 7f72595b7700 1 -- 172.21.15.64:0/3205758722 shutdown_connections 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.511+0000 7f72595b7700 1 --2- 172.21.15.64:0/3205758722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7254106c20 0x7f7254109090 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.511+0000 7f72595b7700 1 --2- 172.21.15.64:0/3205758722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541019c0 0x7f7254103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.511+0000 7f72595b7700 1 --2- 172.21.15.64:0/3205758722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72541042f0 0x7f72541066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.511+0000 7f72595b7700 1 -- 172.21.15.64:0/3205758722 >> 172.21.15.64:0/3205758722 conn(0x7f72540fb3b0 msgr2=0x7f72540fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.512+0000 7f72595b7700 1 -- 172.21.15.64:0/3205758722 shutdown_connections 2024-05-08T22:02:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.512+0000 7f72595b7700 1 -- 172.21.15.64:0/3205758722 wait complete. 2024-05-08T22:02:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 Processor -- start 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 -- start start 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 0x7f7254196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541042f0 0x7f72541974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7254106c20 0x7f725419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419eaa0 con 0x7f72541019c0 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419ebe0 con 0x7f7254106c20 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72595b7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419ed70 con 0x7f72541042f0 2024-05-08T22:02:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 0x7f7254196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 0x7f7254196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55894/0 (socket says 172.21.15.64:55894) 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f7252ffd700 1 -- 172.21.15.64:0/3612643289 learned_addr learned my addr 172.21.15.64:0/3612643289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72537fe700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7254106c20 0x7f725419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.513+0000 7f72527fc700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541042f0 0x7f72541974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f7252ffd700 1 -- 172.21.15.64:0/3612643289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541042f0 msgr2=0x7f72541974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541042f0 0x7f72541974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f7252ffd700 1 -- 172.21.15.64:0/3612643289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7254106c20 msgr2=0x7f725419e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7254106c20 0x7f725419e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f7252ffd700 1 -- 172.21.15.64:0/3612643289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c009d00 con 0x7f72541019c0 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f72527fc700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541042f0 0x7f72541974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f72537fe700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7254106c20 0x7f725419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 0x7f7254196f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f724400d950 tx=0x7f724400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f723bfff700 1 -- 172.21.15.64:0/3612643289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400f890 con 0x7f72541019c0 2024-05-08T22:02:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725419eff0 con 0x7f72541019c0 2024-05-08T22:02:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f723bfff700 1 -- 172.21.15.64:0/3612643289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7244010460 con 0x7f72541019c0 2024-05-08T22:02:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f723bfff700 1 -- 172.21.15.64:0/3612643289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400e610 con 0x7f72541019c0 2024-05-08T22:02:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.514+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725419f470 con 0x7f72541019c0 2024-05-08T22:02:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.517+0000 7f723bfff700 1 -- 172.21.15.64:0/3612643289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f72440108d0 con 0x7f72541019c0 2024-05-08T22:02:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.518+0000 7f723bfff700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f723c0a1120 0x7f723c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.518+0000 7f723bfff700 1 -- 172.21.15.64:0/3612643289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f72440cc660 con 0x7f72541019c0 2024-05-08T22:02:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.518+0000 7f72527fc700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f723c0a1120 0x7f723c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.519+0000 7f72527fc700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f723c0a1120 0x7f723c0a35e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7248011490 tx=0x7f7248009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 Processor -- start 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 -- start start 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf930ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf931cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf932750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf8880f0 con 0x55adbf9322d0 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf885ab0 con 0x55adbf930a80 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7265771b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf8813a0 con 0x55adbf9313e0 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f72537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf932750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f72537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf932750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55900/0 (socket says 172.21.15.64:55900) 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f72537fe700 1 -- 172.21.15.64:0/2226172187 learned_addr learned my addr 172.21.15.64:0/2226172187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f7252ffd700 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf930ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.520+0000 7f72527fc700 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf931cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f7252ffd700 1 -- 172.21.15.64:0/2226172187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 msgr2=0x55adbf931cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f7252ffd700 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf931cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f7252ffd700 1 -- 172.21.15.64:0/2226172187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 msgr2=0x55adbf932750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f7252ffd700 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf932750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f7252ffd700 1 -- 172.21.15.64:0/2226172187 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adbf702350 con 0x55adbf930a80 2024-05-08T22:02:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f72537fe700 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf932750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f7252ffd700 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf930ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f724400b2f0 tx=0x7f724409a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a3516e07e49389 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f72397fa700 1 -- 172.21.15.64:0/2226172187 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440a1f40 con 0x55adbf930a80 2024-05-08T22:02:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f72397fa700 1 -- 172.21.15.64:0/2226172187 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7244097e60 con 0x55adbf930a80 2024-05-08T22:02:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.521+0000 7f72397fa700 1 -- 172.21.15.64:0/2226172187 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440a1ab0 con 0x55adbf930a80 2024-05-08T22:02:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 -- 172.21.15.64:0/2226172187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 msgr2=0x55adbf930ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf930ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f724400b2f0 tx=0x7f724409a1c0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 -- 172.21.15.64:0/2226172187 shutdown_connections 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf930ea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf932750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 --2- 172.21.15.64:0/2226172187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf931cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 -- 172.21.15.64:0/2226172187 >> 172.21.15.64:0/2226172187 conn(0x55adbf88f1b0 msgr2=0x55adbf92d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 -- 172.21.15.64:0/2226172187 shutdown_connections 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.522+0000 7f7265771b80 1 -- 172.21.15.64:0/2226172187 wait complete. 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 -- start start 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 Processor -- start 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf93ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf941e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf81d500 con 0x55adbf9322d0 2024-05-08T22:02:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf81d1a0 con 0x55adbf930a80 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf704640 con 0x55adbf9313e0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7265771b80 10 client.0 _ll_register_callbacks cb 0x55adbf826ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f72537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf941e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f72537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf941e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55912/0 (socket says 172.21.15.64:55912) 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f72537fe700 1 -- 172.21.15.64:0/136802320 learned_addr learned my addr 172.21.15.64:0/136802320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.523+0000 7f72527fc700 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf93ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7252ffd700 1 -- 172.21.15.64:0/136802320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 msgr2=0x55adbf93ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7252ffd700 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf93ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7252ffd700 1 -- 172.21.15.64:0/136802320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 msgr2=0x55adbf941e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7252ffd700 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf941e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7252ffd700 1 -- 172.21.15.64:0/136802320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7254109c80 con 0x55adbf930a80 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7252ffd700 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93a910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7244095fc0 tx=0x7f7244099890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98799713cb774fba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7236ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7236ff5700 1 -- 172.21.15.64:0/136802320 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440a1ab0 con 0x55adbf930a80 2024-05-08T22:02:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7236ff5700 1 -- 172.21.15.64:0/136802320 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f72440a1c10 con 0x55adbf930a80 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adbf7da210 con 0x55adbf930a80 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7236ff5700 1 -- 172.21.15.64:0/136802320 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72440b21a0 con 0x55adbf930a80 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7265771b80 10 client.25574 fetch_fsmap learned FSMap version 39 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.524+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f725419f470 con 0x55adbf930a80 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7236ff5700 1 -- 172.21.15.64:0/136802320 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55adbf7da210 con 0x55adbf930a80 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7236ff5700 1 -- 172.21.15.64:0/136802320 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f72440959d0 con 0x55adbf930a80 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7265771b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7265771b80 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7265771b80 20 client.25574 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7265771b80 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf9462b0 0x55adbf9486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7265771b80 4 client.25574 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adbf9007b0 con 0x55adbf9462b0 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f72527fc700 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf9462b0 0x55adbf9486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f7236ff5700 10 client.25574 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.525+0000 7f72527fc700 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf9462b0 0x55adbf9486a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198f9373dbf55767 server_cookie=a7723b365b54788e in_seq=0 out_seq=0 2024-05-08T22:02:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.526+0000 7f7236ff5700 1 -- 172.21.15.64:0/136802320 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adbf9007b0 con 0x55adbf9462b0 2024-05-08T22:02:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.526+0000 7f7236ff5700 10 client.25574 handle_command_reply: tid=0 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.527+0000 7f7265771b80 1 client.25574 shutdown 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.527+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 msgr2=0x55adbf93a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.527+0000 7f7265771b80 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93a910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7244095fc0 tx=0x7f7244099890 comp rx=0 tx=0).stop 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 shutdown_connections 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93a910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf9462b0 0x55adbf9486a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9322d0 0x55adbf941e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7236ff5700 0 client.25574 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 --2- 172.21.15.64:0/136802320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9313e0 0x55adbf93ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 >> 172.21.15.64:0/136802320 conn(0x55adbf88f1b0 msgr2=0x55adbf92b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 shutdown_connections 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 1 -- 172.21.15.64:0/136802320 wait complete. 2024-05-08T22:02:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.528+0000 7f7265771b80 20 client.25574 trim_cache size 0 max 16384 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.551+0000 7f7265771b80 1 Processor -- start 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 -- start start 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf930a80 0x55adbf942540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 0x55adbf941e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 0x55adbf9326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf9007b0 con 0x55adbf9313e0 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf7da210 con 0x55adbf9322d0 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f7265771b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf9267c0 con 0x55adbf930a80 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f72537fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 0x55adbf9326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f72537fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 0x55adbf9326b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:50698/0 (socket says 172.21.15.64:50698) 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.552+0000 7f72537fe700 1 -- 172.21.15.64:0/3225850142 learned_addr learned my addr 172.21.15.64:0/3225850142 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf930a80 0x55adbf942540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f72537fe700 1 -- 172.21.15.64:0/3225850142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf930a80 msgr2=0x55adbf942540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f72537fe700 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf930a80 0x55adbf942540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f72537fe700 1 -- 172.21.15.64:0/3225850142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 msgr2=0x55adbf941e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f72537fe700 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 0x55adbf941e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f72537fe700 1 -- 172.21.15.64:0/3225850142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adbf6aeb00 con 0x55adbf9322d0 2024-05-08T22:02:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f72537fe700 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 0x55adbf9326b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f724c009fd0 tx=0x7f724c007590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca42a35090f1a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.556+0000 7f7239ffb700 1 -- 172.21.15.64:0/3225850142 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00e040 con 0x55adbf9322d0 2024-05-08T22:02:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7239ffb700 1 -- 172.21.15.64:0/3225850142 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f724c003a00 con 0x55adbf9322d0 2024-05-08T22:02:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7239ffb700 1 -- 172.21.15.64:0/3225850142 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0499d0 con 0x55adbf9322d0 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 -- 172.21.15.64:0/3225850142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 msgr2=0x55adbf9326b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 0x55adbf9326b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f724c009fd0 tx=0x7f724c007590 comp rx=0 tx=0).stop 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 -- 172.21.15.64:0/3225850142 shutdown_connections 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf9322d0 0x55adbf9326b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf930a80 0x55adbf942540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 --2- 172.21.15.64:0/3225850142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 0x55adbf941e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 -- 172.21.15.64:0/3225850142 >> 172.21.15.64:0/3225850142 conn(0x55adbf88f1b0 msgr2=0x55adbf939e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 -- 172.21.15.64:0/3225850142 shutdown_connections 2024-05-08T22:02:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.557+0000 7f7265771b80 1 -- 172.21.15.64:0/3225850142 wait complete. 2024-05-08T22:02:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 -- start start 2024-05-08T22:02:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 Processor -- start 2024-05-08T22:02:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 0x55adbf93fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9322d0 0x55adbf940d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf941240 con 0x55adbf9313e0 2024-05-08T22:02:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf9413b0 con 0x55adbf930a80 2024-05-08T22:02:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adbf941570 con 0x55adbf9322d0 2024-05-08T22:02:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.558+0000 7f7265771b80 10 client.0 _ll_register_callbacks cb 0x55adbf826ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.564+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.564+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:50704/0 (socket says 172.21.15.64:50704) 2024-05-08T22:02:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.564+0000 7f7252ffd700 1 -- 172.21.15.64:0/3714337397 learned_addr learned my addr 172.21.15.64:0/3714337397 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.565+0000 7f7252ffd700 1 -- 172.21.15.64:0/3714337397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9322d0 msgr2=0x55adbf940d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:02:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.565+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9322d0 0x55adbf940d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.565+0000 7f7252ffd700 1 -- 172.21.15.64:0/3714337397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 msgr2=0x55adbf93fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.565+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 0x55adbf93fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.565+0000 7f7252ffd700 1 -- 172.21.15.64:0/3714337397 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adbf847500 con 0x55adbf930a80 2024-05-08T22:02:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.566+0000 7f7252ffd700 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93f800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7244099920 tx=0x7f724409ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6797d0b6a1b16ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.566+0000 7f7236ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:02:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.566+0000 7f7236ff5700 1 -- 172.21.15.64:0/3714337397 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244098fb0 con 0x55adbf930a80 2024-05-08T22:02:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.566+0000 7f7236ff5700 1 -- 172.21.15.64:0/3714337397 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7244099110 con 0x55adbf930a80 2024-05-08T22:02:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.566+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7244097e60 con 0x55adbf930a80 2024-05-08T22:02:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.566+0000 7f7236ff5700 1 -- 172.21.15.64:0/3714337397 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72440a1ab0 con 0x55adbf930a80 2024-05-08T22:02:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.577+0000 7f7265771b80 10 client.25580 fetch_fsmap learned FSMap version 39 2024-05-08T22:02:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.577+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55adbf706c70 con 0x55adbf930a80 2024-05-08T22:02:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.583+0000 7f7236ff5700 1 -- 172.21.15.64:0/3714337397 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7244097e60 con 0x55adbf930a80 2024-05-08T22:02:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7236ff5700 1 -- 172.21.15.64:0/3714337397 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f72440a6030 con 0x55adbf930a80 2024-05-08T22:02:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7265771b80 10 client.25580 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7265771b80 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7265771b80 20 client.25580 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7265771b80 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf99cec0 0x55adbf99f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7265771b80 4 client.25580 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55adbf879e60 con 0x55adbf99cec0 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f72527fc700 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf99cec0 0x55adbf99f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.584+0000 7f7236ff5700 10 client.25580 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.585+0000 7f72527fc700 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf99cec0 0x55adbf99f2e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5e201c87a63a4a server_cookie=87a842b150c630db in_seq=0 out_seq=0 2024-05-08T22:02:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7236ff5700 1 -- 172.21.15.64:0/3714337397 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55adbf879e60 con 0x55adbf99cec0 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7236ff5700 10 client.25580 handle_command_reply: tid=0 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 client.25580 shutdown 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 msgr2=0x55adbf93f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93f800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7244099920 tx=0x7f724409ae20 comp rx=0 tx=0).stop 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 shutdown_connections 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55adbf930a80 0x55adbf93f800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55adbf99cec0 0x55adbf99f2e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adbf9322d0 0x55adbf940d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 --2- 172.21.15.64:0/3714337397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adbf9313e0 0x55adbf93fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 >> 172.21.15.64:0/3714337397 conn(0x55adbf88f1b0 msgr2=0x55adbf9388c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7236ff5700 0 client.25580 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 shutdown_connections 2024-05-08T22:02:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.598+0000 7f7265771b80 1 -- 172.21.15.64:0/3714337397 wait complete. 2024-05-08T22:02:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f7265771b80 20 client.25580 trim_cache size 0 max 16384 2024-05-08T22:02:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:02:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:02:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:02:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f723c0a1120 msgr2=0x7f723c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f723c0a1120 0x7f723c0a35e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7248011490 tx=0x7f7248009260 comp rx=0 tx=0).stop 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 msgr2=0x7f7254196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 0x7f7254196f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f724400d950 tx=0x7f724400dc60 comp rx=0 tx=0).stop 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 shutdown_connections 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f723c0a1120 0x7f723c0a35e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7254106c20 0x7f725419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72541019c0 0x7f7254196f70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 --2- 172.21.15.64:0/3612643289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72541042f0 0x7f72541974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 >> 172.21.15.64:0/3612643289 conn(0x7f72540fb3b0 msgr2=0x7f7254102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 shutdown_connections 2024-05-08T22:02:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:35.599+0000 7f72595b7700 1 -- 172.21.15.64:0/3612643289 wait complete. 2024-05-08T22:02:35.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:35 smithi186 ceph-mon[46619]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-05-08T22:02:35.969 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:35 smithi064 ceph-mon[42500]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-05-08T22:02:36.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:35 smithi160 ceph-mon[46962]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-05-08T22:02:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 465.0 s, 1927.6 tps, lat 51.654 ms stddev 60.387 2024-05-08T22:02:37.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:37 smithi186 ceph-mon[46619]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 806 op/s 2024-05-08T22:02:37.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:37 smithi064 ceph-mon[42500]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 806 op/s 2024-05-08T22:02:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:37 smithi160 ceph-mon[46962]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 806 op/s 2024-05-08T22:02:39.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:39 smithi186 ceph-mon[46619]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 793 op/s 2024-05-08T22:02:39.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:39 smithi064 ceph-mon[42500]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 793 op/s 2024-05-08T22:02:40.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:39 smithi160 ceph-mon[46962]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 793 op/s 2024-05-08T22:02:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 470.0 s, 1684.4 tps, lat 58.323 ms stddev 66.470 2024-05-08T22:02:41.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:41 smithi064 ceph-mon[42500]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 821 op/s 2024-05-08T22:02:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:41 smithi160 ceph-mon[46962]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 821 op/s 2024-05-08T22:02:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:41 smithi186 ceph-mon[46619]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 821 op/s 2024-05-08T22:02:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:43 smithi064 ceph-mon[42500]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 679 op/s 2024-05-08T22:02:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:43 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:02:44.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:43 smithi160 ceph-mon[46962]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 679 op/s 2024-05-08T22:02:44.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:43 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:02:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:43 smithi186 ceph-mon[46619]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 679 op/s 2024-05-08T22:02:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:43 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:02:45.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:45 smithi064 ceph-mon[42500]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 761 op/s 2024-05-08T22:02:46.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:45 smithi160 ceph-mon[46962]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 761 op/s 2024-05-08T22:02:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:45 smithi186 ceph-mon[46619]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 761 op/s 2024-05-08T22:02:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 475.0 s, 1754.5 tps, lat 57.564 ms stddev 66.491 2024-05-08T22:02:48.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:47 smithi160 ceph-mon[46962]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 814 op/s 2024-05-08T22:02:48.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:47 smithi186 ceph-mon[46619]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 814 op/s 2024-05-08T22:02:48.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:47 smithi064 ceph-mon[42500]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 814 op/s 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 -- 172.21.15.64:0/3575413277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a18101990 msgr2=0x7f0a18108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/3575413277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a18101990 0x7f0a18108240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009990 tx=0x7f0a10009ca0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 -- 172.21.15.64:0/3575413277 shutdown_connections 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/3575413277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a18101990 0x7f0a18108240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/3575413277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a180ffe00 0x7f0a18100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/3575413277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a18100ff0 0x7f0a18101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 -- 172.21.15.64:0/3575413277 >> 172.21.15.64:0/3575413277 conn(0x7f0a180fb390 msgr2=0x7f0a180fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 -- 172.21.15.64:0/3575413277 shutdown_connections 2024-05-08T22:02:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 -- 172.21.15.64:0/3575413277 wait complete. 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.767+0000 7f0a1fee1700 1 Processor -- start 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 -- start start 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 0x7f0a18194c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18100ff0 0x7f0a18195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a18101990 0x7f0a1819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1819c770 con 0x7f0a18100ff0 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1819c8b0 con 0x7f0a18101990 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1fee1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1819ca40 con 0x7f0a180ffe00 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 0x7f0a18194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 0x7f0a18194c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42456/0 (socket says 172.21.15.64:42456) 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/1844145848 learned_addr learned my addr 172.21.15.64:0/1844145848 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18100ff0 0x7f0a18195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a18101990 0x7f0a1819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/1844145848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a18101990 msgr2=0x7f0a1819c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a18101990 0x7f0a1819c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18100ff0 msgr2=0x7f0a18195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18100ff0 0x7f0a18195180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/1844145848 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a100095f0 con 0x7f0a180ffe00 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.768+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18100ff0 0x7f0a18195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.769+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 0x7f0a18194c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c007e00 tx=0x7f0a0c00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.769+0000 7f0a16ffd700 1 -- 172.21.15.64:0/1844145848 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00ad40 con 0x7f0a180ffe00 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.769+0000 7f0a16ffd700 1 -- 172.21.15.64:0/1844145848 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0c0128e0 con 0x7f0a180ffe00 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.769+0000 7f0a16ffd700 1 -- 172.21.15.64:0/1844145848 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c011810 con 0x7f0a180ffe00 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.769+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1819cd20 con 0x7f0a180ffe00 2024-05-08T22:02:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.769+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1819d180 con 0x7f0a180ffe00 2024-05-08T22:02:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.774+0000 7f0a2c09bb80 1 Processor -- start 2024-05-08T22:02:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.774+0000 7f0a16ffd700 1 -- 172.21.15.64:0/1844145848 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0a0c012a50 con 0x7f0a180ffe00 2024-05-08T22:02:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 -- start start 2024-05-08T22:02:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b037ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0383e0 0x55be0b038cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b039750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0af8f0f0 con 0x55be0b0392d0 2024-05-08T22:02:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0af8cab0 con 0x55be0b0383e0 2024-05-08T22:02:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0af883a0 con 0x55be0b037a80 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1e47e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b037ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1dc7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0383e0 0x55be0b038cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1d47c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b039750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1e47e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b037ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42468/0 (socket says 172.21.15.64:42468) 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1e47e700 1 -- 172.21.15.64:0/890020963 learned_addr learned my addr 172.21.15.64:0/890020963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a16ffd700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0a080a11a0 0x7f0a080a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/890020963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 msgr2=0x55be0b037ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b037ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/890020963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 msgr2=0x55be0b039750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.775+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b039750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/890020963 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0ae09350 con 0x55be0b0383e0 2024-05-08T22:02:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b037ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b039750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a16ffd700 1 -- 172.21.15.64:0/1844145848 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a0c0cc970 con 0x7f0a180ffe00 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0a080a11a0 0x7f0a080a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0383e0 0x55be0b038cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0967e0 tx=0x7f0a0c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0786d44c2bcf937 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a03fff700 1 -- 172.21.15.64:0/890020963 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0cfea0 con 0x55be0b0383e0 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a03fff700 1 -- 172.21.15.64:0/890020963 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0c01c830 con 0x55be0b0383e0 2024-05-08T22:02:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.776+0000 7f0a03fff700 1 -- 172.21.15.64:0/890020963 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c015170 con 0x55be0b0383e0 2024-05-08T22:02:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/890020963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0383e0 msgr2=0x55be0b038cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0383e0 0x55be0b038cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0967e0 tx=0x7f0a0c021000 comp rx=0 tx=0).stop 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/890020963 shutdown_connections 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b039750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b037ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/890020963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0383e0 0x55be0b038cf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/890020963 >> 172.21.15.64:0/890020963 conn(0x55be0af961b0 msgr2=0x55be0b034440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/890020963 shutdown_connections 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0a080a11a0 0x7f0a080a3660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000d7d0 tx=0x7f0a10017040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.777+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/890020963 wait complete. 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 -- start start 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 Processor -- start 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 0x55be0b04a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b0515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0af24500 con 0x55be0b0392d0 2024-05-08T22:02:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0af241a0 con 0x55be0b037a80 2024-05-08T22:02:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.778+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0ae0b640 con 0x55be0b0383e0 2024-05-08T22:02:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a2c09bb80 10 client.0 _ll_register_callbacks cb 0x55be0af2dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 0x55be0b04a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 0x55be0b04a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42472/0 (socket says 172.21.15.64:42472) 2024-05-08T22:02:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 -- 172.21.15.64:0/3618680798 learned_addr learned my addr 172.21.15.64:0/3618680798 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 -- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 msgr2=0x55be0b04a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 -- 172.21.15.64:0/3618680798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 msgr2=0x55be0b0515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b0515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.779+0000 7f0a1d47c700 1 -- 172.21.15.64:0/3618680798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0400bb80 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.780+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 0x55be0b04a580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009b00 tx=0x7f0a04009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc9f9c3ad34f19f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.780+0000 7f0a017fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.780+0000 7f0a017fa700 1 -- 172.21.15.64:0/3618680798 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04007ac0 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.780+0000 7f0a017fa700 1 -- 172.21.15.64:0/3618680798 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a04007c20 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.780+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be0aee1210 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a017fa700 1 -- 172.21.15.64:0/3618680798 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a0400b440 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 10 client.25477 fetch_fsmap learned FSMap version 41 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f0a1819d180 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a017fa700 1 -- 172.21.15.64:0/3618680798 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be0aee1210 con 0x55be0b0383e0 2024-05-08T22:02:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a017fa700 1 -- 172.21.15.64:0/3618680798 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0a0400b040 con 0x55be0b0383e0 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 10 client.25477 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 20 client.25477 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b067050 0x55be0b069470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 4 client.25477 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.781+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be0b0077b0 con 0x55be0b067050 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.782+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b067050 0x55be0b069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.782+0000 7f0a017fa700 10 client.25477 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.782+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b067050 0x55be0b069470 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107b0759b8533eed server_cookie=cffa2cc1a33b5a41 in_seq=0 out_seq=0 2024-05-08T22:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a017fa700 1 -- 172.21.15.64:0/3618680798 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be0b0077b0 con 0x55be0b067050 2024-05-08T22:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a017fa700 10 client.25477 handle_command_reply: tid=0 2024-05-08T22:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 client.25477 shutdown 2024-05-08T22:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 msgr2=0x55be0b04a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 0x55be0b04a580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009b00 tx=0x7f0a04009ec0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 shutdown_connections 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b067050 0x55be0b069470 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0392d0 0x55be0b0515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/3618680798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0383e0 0x55be0b04a580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 >> 172.21.15.64:0/3618680798 conn(0x55be0af961b0 msgr2=0x55be0b032860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a017fa700 0 client.25477 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 shutdown_connections 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/3618680798 wait complete. 2024-05-08T22:02:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.783+0000 7f0a2c09bb80 20 client.25477 trim_cache size 0 max 16384 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 Processor -- start 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 -- start start 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b0396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b0514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0392d0 0x55be0b048d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0b038a60 con 0x55be0b0383e0 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0b037680 con 0x55be0b0392d0 2024-05-08T22:02:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.803+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0b04acc0 con 0x55be0b037a80 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b0514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b0514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40484/0 (socket says 172.21.15.64:40484) 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 -- 172.21.15.64:0/57883244 learned_addr learned my addr 172.21.15.64:0/57883244 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b0396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0392d0 0x55be0b048d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 -- 172.21.15.64:0/57883244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 msgr2=0x55be0b0396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b0396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 -- 172.21.15.64:0/57883244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0392d0 msgr2=0x55be0b048d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0392d0 0x55be0b048d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 -- 172.21.15.64:0/57883244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0adb5b00 con 0x55be0b0383e0 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b0396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b0514e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400bb50 tx=0x7f0a040186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38e507eda575925 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.804+0000 7f0a14ff9700 1 -- 172.21.15.64:0/57883244 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04018bb0 con 0x55be0b0383e0 2024-05-08T22:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a14ff9700 1 -- 172.21.15.64:0/57883244 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a04013040 con 0x55be0b0383e0 2024-05-08T22:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a14ff9700 1 -- 172.21.15.64:0/57883244 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04007d10 con 0x55be0b0383e0 2024-05-08T22:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/57883244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 msgr2=0x55be0b0514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b0514e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400bb50 tx=0x7f0a040186a0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/57883244 shutdown_connections 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b0392d0 0x55be0b048d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b037a80 0x55be0b0396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/57883244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b0514e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.805+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/57883244 >> 172.21.15.64:0/57883244 conn(0x55be0af961b0 msgr2=0x55be0b03f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.806+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/57883244 shutdown_connections 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.806+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/57883244 wait complete. 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.806+0000 7f0a2c09bb80 1 -- start start 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.806+0000 7f0a2c09bb80 1 Processor -- start 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b04eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0392d0 0x55be0b0bb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0b04f220 con 0x55be0b0383e0 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0b04f390 con 0x55be0b037a80 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0b040590 con 0x55be0b0392d0 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a2c09bb80 10 client.0 _ll_register_callbacks cb 0x55be0af2dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35444/0 (socket says 172.21.15.64:35444) 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/718329713 learned_addr learned my addr 172.21.15.64:0/718329713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b04eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0392d0 0x55be0b0bb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/718329713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0392d0 msgr2=0x55be0b0bb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0392d0 0x55be0b0bb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/718329713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 msgr2=0x55be0b04eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b04eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1dc7d700 1 -- 172.21.15.64:0/718329713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be0ae09350 con 0x55be0b037a80 2024-05-08T22:02:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.807+0000 7f0a1e47e700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0392d0 0x55be0b0bb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a1dc7d700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04e760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c099a70 tx=0x7f0a0c00af30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282c6b2357aa8d9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a017fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a017fa700 1 -- 172.21.15.64:0/718329713 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c078ea0 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a017fa700 1 -- 172.21.15.64:0/718329713 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a0c0a21f0 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a0c01c830 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a017fa700 1 -- 172.21.15.64:0/718329713 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a0c0b8170 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a2c09bb80 10 client.25598 fetch_fsmap learned FSMap version 41 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f0a1819d180 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a017fa700 1 -- 172.21.15.64:0/718329713 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0c07c080 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.808+0000 7f0a017fa700 1 -- 172.21.15.64:0/718329713 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0a0c0ccd20 con 0x55be0b037a80 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a2c09bb80 10 client.25598 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a2c09bb80 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a2c09bb80 20 client.25598 populate_metadata read hostname 'smithi064' 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b041d20 0x55be0b044140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a2c09bb80 4 client.25598 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:02:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be0af80e60 con 0x55be0b041d20 2024-05-08T22:02:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b041d20 0x55be0b044140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a017fa700 10 client.25598 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.809+0000 7f0a1d47c700 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b041d20 0x55be0b044140 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492df2e22a932cfd server_cookie=f030e455f2fdcb5f in_seq=0 out_seq=0 2024-05-08T22:02:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.822+0000 7f0a017fa700 1 -- 172.21.15.64:0/718329713 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be0af80e60 con 0x55be0b041d20 2024-05-08T22:02:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.822+0000 7f0a017fa700 10 client.25598 handle_command_reply: tid=0 2024-05-08T22:02:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.822+0000 7f0a2c09bb80 1 client.25598 shutdown 2024-05-08T22:02:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 msgr2=0x55be0b04e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04e760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c099a70 tx=0x7f0a0c00af30 comp rx=0 tx=0).stop 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 shutdown_connections 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be0b037a80 0x55be0b04e760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be0b041d20 0x55be0b044140 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be0b0392d0 0x55be0b0bb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 --2- 172.21.15.64:0/718329713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0b0383e0 0x55be0b04eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 >> 172.21.15.64:0/718329713 conn(0x55be0af961b0 msgr2=0x55be0b048d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a017fa700 0 client.25598 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 shutdown_connections 2024-05-08T22:02:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.823+0000 7f0a2c09bb80 1 -- 172.21.15.64:0/718329713 wait complete. 2024-05-08T22:02:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.824+0000 7f0a2c09bb80 20 client.25598 trim_cache size 0 max 16384 2024-05-08T22:02:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:02:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:02:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:02:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0a080a11a0 msgr2=0x7f0a080a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0a080a11a0 0x7f0a080a3660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000d7d0 tx=0x7f0a10017040 comp rx=0 tx=0).stop 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 msgr2=0x7f0a18194c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 0x7f0a18194c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c007e00 tx=0x7f0a0c00a340 comp rx=0 tx=0).stop 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 shutdown_connections 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f0a080a11a0 0x7f0a080a3660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a18101990 0x7f0a1819c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a180ffe00 0x7f0a18194c40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 --2- 172.21.15.64:0/1844145848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18100ff0 0x7f0a18195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 >> 172.21.15.64:0/1844145848 conn(0x7f0a180fb390 msgr2=0x7f0a18104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 shutdown_connections 2024-05-08T22:02:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:02:48.825+0000 7f0a1fee1700 1 -- 172.21.15.64:0/1844145848 wait complete. 2024-05-08T22:02:49.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:48 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:02:49.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:48 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:02:49.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:48 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:02:50.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:49 smithi160 ceph-mon[46962]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 752 op/s 2024-05-08T22:02:50.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:49 smithi186 ceph-mon[46619]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 752 op/s 2024-05-08T22:02:50.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:49 smithi064 ceph-mon[42500]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 752 op/s 2024-05-08T22:02:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 480.0 s, 1906.2 tps, lat 52.895 ms stddev 59.099 2024-05-08T22:02:52.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:51 smithi160 ceph-mon[46962]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 787 op/s 2024-05-08T22:02:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:51 smithi186 ceph-mon[46619]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 787 op/s 2024-05-08T22:02:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:51 smithi064 ceph-mon[42500]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 787 op/s 2024-05-08T22:02:54.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:53 smithi160 ceph-mon[46962]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 715 op/s 2024-05-08T22:02:54.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:53 smithi186 ceph-mon[46619]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 715 op/s 2024-05-08T22:02:54.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:53 smithi064 ceph-mon[42500]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 715 op/s 2024-05-08T22:02:56.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:55 smithi160 ceph-mon[46962]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 20 MiB/s wr, 788 op/s 2024-05-08T22:02:56.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:55 smithi186 ceph-mon[46619]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 20 MiB/s wr, 788 op/s 2024-05-08T22:02:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:55 smithi064 ceph-mon[42500]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 20 MiB/s wr, 788 op/s 2024-05-08T22:02:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 485.0 s, 1614.9 tps, lat 59.576 ms stddev 68.497 2024-05-08T22:02:58.145 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:57 smithi186 ceph-mon[46619]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 19 MiB/s wr, 790 op/s 2024-05-08T22:02:58.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:57 smithi064 ceph-mon[42500]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 19 MiB/s wr, 790 op/s 2024-05-08T22:02:58.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:57 smithi160 ceph-mon[46962]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 19 MiB/s wr, 790 op/s 2024-05-08T22:03:00.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:02:59 smithi186 ceph-mon[46619]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-05-08T22:03:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:02:59 smithi064 ceph-mon[42500]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-05-08T22:03:00.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:02:59 smithi160 ceph-mon[46962]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-05-08T22:03:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 490.0 s, 1546.6 tps, lat 66.665 ms stddev 115.009 2024-05-08T22:03:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:01.995+0000 7f7a39beb700 1 -- 172.21.15.64:0/3921537343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410a910 msgr2=0x7f7a3410cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:01.995+0000 7f7a39beb700 1 --2- 172.21.15.64:0/3921537343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410a910 0x7f7a3410cda0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009990 tx=0x7f7a24009ca0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 -- 172.21.15.64:0/3921537343 shutdown_connections 2024-05-08T22:03:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 --2- 172.21.15.64:0/3921537343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410d2e0 0x7f7a3410d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 --2- 172.21.15.64:0/3921537343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a34107fa0 0x7f7a3410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 --2- 172.21.15.64:0/3921537343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410a910 0x7f7a3410cda0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 -- 172.21.15.64:0/3921537343 >> 172.21.15.64:0/3921537343 conn(0x7f7a3406df30 msgr2=0x7f7a34070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 -- 172.21.15.64:0/3921537343 shutdown_connections 2024-05-08T22:03:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.000+0000 7f7a39beb700 1 -- 172.21.15.64:0/3921537343 wait complete. 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.001+0000 7f7a39beb700 1 Processor -- start 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.001+0000 7f7a39beb700 1 -- start start 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a39beb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a34107fa0 0x7f7a341a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a39beb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 0x7f7a341a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a39beb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410d2e0 0x7f7a341ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a39beb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341ad8e0 con 0x7f7a3410d2e0 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a39beb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341ada50 con 0x7f7a3410a910 2024-05-08T22:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a39beb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341adc10 con 0x7f7a34107fa0 2024-05-08T22:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 0x7f7a341a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 0x7f7a341a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:39990/0 (socket says 172.21.15.64:39990) 2024-05-08T22:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 -- 172.21.15.64:0/527660325 learned_addr learned my addr 172.21.15.64:0/527660325 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 -- 172.21.15.64:0/527660325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a34107fa0 msgr2=0x7f7a341a5df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a34107fa0 0x7f7a341a5df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 -- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410d2e0 msgr2=0x7f7a341ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410d2e0 0x7f7a341ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.002+0000 7f7a32ffd700 1 -- 172.21.15.64:0/527660325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a240095f0 con 0x7f7a3410a910 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.003+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 0x7f7a341a6330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a240036f0 tx=0x7f7a24003d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.004+0000 7f7a30ff9700 1 -- 172.21.15.64:0/527660325 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2401d070 con 0x7f7a3410a910 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.004+0000 7f7a30ff9700 1 -- 172.21.15.64:0/527660325 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a2400f460 con 0x7f7a3410a910 2024-05-08T22:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.004+0000 7f7a30ff9700 1 -- 172.21.15.64:0/527660325 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24017650 con 0x7f7a3410a910 2024-05-08T22:03:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.004+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a341ade90 con 0x7f7a3410a910 2024-05-08T22:03:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.004+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a341ae200 con 0x7f7a3410a910 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.008+0000 7f7a45da5b80 1 Processor -- start 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.008+0000 7f7a30ff9700 1 -- 172.21.15.64:0/527660325 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7a2400f5d0 con 0x7f7a3410a910 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.009+0000 7f7a30ff9700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7a1c0a1320 0x7f7a1c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.009+0000 7f7a30ff9700 1 -- 172.21.15.64:0/527660325 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a240cdb20 con 0x7f7a3410a910 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 -- start start 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f6aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6b3f0 0x55a401f6bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6c2e0 0x55a401f6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401eb10f0 con 0x55a401f6b3f0 2024-05-08T22:03:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401eaeab0 con 0x55a401f6aa90 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401eaa3a0 con 0x55a401f6c2e0 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f6aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f6aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:39992/0 (socket says 172.21.15.64:39992) 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 -- 172.21.15.64:0/3431943871 learned_addr learned my addr 172.21.15.64:0/3431943871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 -- 172.21.15.64:0/3431943871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6c2e0 msgr2=0x55a401f6c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6c2e0 0x55a401f6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 -- 172.21.15.64:0/3431943871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6b3f0 msgr2=0x55a401f6bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6b3f0 0x55a401f6bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.010+0000 7f7a337fe700 1 -- 172.21.15.64:0/3431943871 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a401d2b350 con 0x55a401f6aa90 2024-05-08T22:03:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.011+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f6aeb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009d50 tx=0x7f7a2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2547f68399f3360d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.011+0000 7f7a19ffb700 1 -- 172.21.15.64:0/3431943871 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00e070 con 0x55a401f6aa90 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.011+0000 7f7a19ffb700 1 -- 172.21.15.64:0/3431943871 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a2c00bbd0 con 0x55a401f6aa90 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.011+0000 7f7a19ffb700 1 -- 172.21.15.64:0/3431943871 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0115f0 con 0x55a401f6aa90 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3431943871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 msgr2=0x55a401f6aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f6aeb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009d50 tx=0x7f7a2c00b330 comp rx=0 tx=0).stop 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a33fff700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7a1c0a1320 0x7f7a1c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3431943871 shutdown_connections 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f6aeb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6c2e0 0x55a401f6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3431943871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6b3f0 0x55a401f6bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3431943871 >> 172.21.15.64:0/3431943871 conn(0x55a401eb81b0 msgr2=0x55a401f56440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3431943871 shutdown_connections 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.012+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3431943871 wait complete. 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 -- start start 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 Processor -- start 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f74e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6c2e0 0x55a401f7be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401e46500 con 0x55a401f6c2e0 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401e461a0 con 0x55a401f6aa90 2024-05-08T22:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401d2d640 con 0x55a401f6b3f0 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.013+0000 7f7a45da5b80 10 client.0 _ll_register_callbacks cb 0x55a401e4fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f748c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40004/0 (socket says 172.21.15.64:40004) 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 -- 172.21.15.64:0/2390122027 learned_addr learned my addr 172.21.15.64:0/2390122027 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 -- 172.21.15.64:0/2390122027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 msgr2=0x55a401f74e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f74e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 -- 172.21.15.64:0/2390122027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6c2e0 msgr2=0x55a401f7be40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6c2e0 0x55a401f7be40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 -- 172.21.15.64:0/2390122027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c009a00 con 0x55a401f6aa90 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a337fe700 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f748c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c011640 tx=0x7f7a2c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756a2985c3f1536b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a12ffd700 1 -- 172.21.15.64:0/2390122027 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00e040 con 0x55a401f6aa90 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a12ffd700 1 -- 172.21.15.64:0/2390122027 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a2c011c80 con 0x55a401f6aa90 2024-05-08T22:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a401e03210 con 0x55a401f6aa90 2024-05-08T22:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a12ffd700 1 -- 172.21.15.64:0/2390122027 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a2c012a00 con 0x55a401f6aa90 2024-05-08T22:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a45da5b80 10 client.25607 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.014+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a401cd7b00 con 0x55a401f6aa90 2024-05-08T22:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a12ffd700 1 -- 172.21.15.64:0/2390122027 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a401e03210 con 0x55a401f6aa90 2024-05-08T22:03:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a12ffd700 1 -- 172.21.15.64:0/2390122027 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7a2c01a380 con 0x55a401f6aa90 2024-05-08T22:03:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a45da5b80 10 client.25607 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a45da5b80 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a45da5b80 20 client.25607 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401f80670 0x55a401f82ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a45da5b80 4 client.25607 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a401f297b0 con 0x55a401f80670 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.015+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401f80670 0x55a401f82ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.016+0000 7f7a33fff700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7a1c0a1320 0x7f7a1c0a37e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55a401f6bac0 tx=0x7f7a28007b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.016+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401f80670 0x55a401f82ab0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108d29b50525fff3 server_cookie=f5c4e10d4a16c4a3 in_seq=0 out_seq=0 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.016+0000 7f7a12ffd700 10 client.25607 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a12ffd700 1 -- 172.21.15.64:0/2390122027 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a401f297b0 con 0x55a401f80670 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a12ffd700 10 client.25607 handle_command_reply: tid=0 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 client.25607 shutdown 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 msgr2=0x55a401f748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f748c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c011640 tx=0x7f7a2c011720 comp rx=0 tx=0).stop 2024-05-08T22:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 shutdown_connections 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6aa90 0x55a401f748c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401f80670 0x55a401f82ab0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6c2e0 0x55a401f7be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2390122027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f74e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a12ffd700 0 client.25607 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 >> 172.21.15.64:0/2390122027 conn(0x55a401eb81b0 msgr2=0x55a401f54860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 shutdown_connections 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.017+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2390122027 wait complete. 2024-05-08T22:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.018+0000 7f7a45da5b80 20 client.25607 trim_cache size 0 max 16384 2024-05-08T22:03:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 Processor -- start 2024-05-08T22:03:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 -- start start 2024-05-08T22:03:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f7be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f6c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401f297b0 con 0x55a401f6aa90 2024-05-08T22:03:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401e03210 con 0x55a401f6c2e0 2024-05-08T22:03:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401f7cfc0 con 0x55a401f6b3f0 2024-05-08T22:03:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a33fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f6c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a33fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f6c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40018/0 (socket says 172.21.15.64:40018) 2024-05-08T22:03:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a33fff700 1 -- 172.21.15.64:0/2115534166 learned_addr learned my addr 172.21.15.64:0/2115534166 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f7be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.037+0000 7f7a337fe700 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a33fff700 1 -- 172.21.15.64:0/2115534166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 msgr2=0x55a401f7be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a33fff700 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f7be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a33fff700 1 -- 172.21.15.64:0/2115534166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 msgr2=0x55a401f7c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a33fff700 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f7c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a33fff700 1 -- 172.21.15.64:0/2115534166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a401e70500 con 0x55a401f6c2e0 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a33fff700 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f6c6c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a280169b0 tx=0x7f7a2800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c688ca13631b72a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a1a7fc700 1 -- 172.21.15.64:0/2115534166 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2801b070 con 0x55a401f6c2e0 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a1a7fc700 1 -- 172.21.15.64:0/2115534166 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a28006e70 con 0x55a401f6c2e0 2024-05-08T22:03:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.038+0000 7f7a1a7fc700 1 -- 172.21.15.64:0/2115534166 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28003960 con 0x55a401f6c2e0 2024-05-08T22:03:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2115534166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 msgr2=0x55a401f6c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f6c6c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a280169b0 tx=0x7f7a2800f000 comp rx=0 tx=0).stop 2024-05-08T22:03:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2115534166 shutdown_connections 2024-05-08T22:03:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f6c6c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f7c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/2115534166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f7be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2115534166 >> 172.21.15.64:0/2115534166 conn(0x55a401eb81b0 msgr2=0x55a401f73dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2115534166 shutdown_connections 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 -- 172.21.15.64:0/2115534166 wait complete. 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.039+0000 7f7a45da5b80 1 -- start start 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 Processor -- start 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f79950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f79e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f7aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401f7b3e0 con 0x55a401f6aa90 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401f7b550 con 0x55a401f6c2e0 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a401f7b710 con 0x55a401f6b3f0 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a45da5b80 10 client.0 _ll_register_callbacks cb 0x55a401e4fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a33fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f7aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a33fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f7aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40026/0 (socket says 172.21.15.64:40026) 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a33fff700 1 -- 172.21.15.64:0/3969011378 learned_addr learned my addr 172.21.15.64:0/3969011378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f79950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.040+0000 7f7a32ffd700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f79e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a33fff700 1 -- 172.21.15.64:0/3969011378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 msgr2=0x55a401f79e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a33fff700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f79e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a33fff700 1 -- 172.21.15.64:0/3969011378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 msgr2=0x55a401f79950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a33fff700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f79950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a33fff700 1 -- 172.21.15.64:0/3969011378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a28016660 con 0x55a401f6c2e0 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f79950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a33fff700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f7aea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a28004920 tx=0x7f7a28005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84747672e2492801 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.041+0000 7f7a12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a12ffd700 1 -- 172.21.15.64:0/3969011378 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2801b070 con 0x55a401f6c2e0 2024-05-08T22:03:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a12ffd700 1 -- 172.21.15.64:0/3969011378 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a28003720 con 0x55a401f6c2e0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a2c00bbd0 con 0x55a401f6c2e0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a12ffd700 1 -- 172.21.15.64:0/3969011378 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a280038e0 con 0x55a401f6c2e0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a45da5b80 10 client.25613 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a401d2fc70 con 0x55a401f6c2e0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.043+0000 7f7a12ffd700 1 -- 172.21.15.64:0/3969011378 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a2c00bbd0 con 0x55a401f6c2e0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a12ffd700 1 -- 172.21.15.64:0/3969011378 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7a28007740 con 0x55a401f6c2e0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a45da5b80 10 client.25613 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a45da5b80 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a45da5b80 20 client.25613 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401fd78a0 0x55a401fd9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a45da5b80 4 client.25613 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a401ea2e60 con 0x55a401fd78a0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401fd78a0 0x55a401fd9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a12ffd700 10 client.25613 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.044+0000 7f7a337fe700 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401fd78a0 0x55a401fd9cc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7645c69fe4d10bd0 server_cookie=d964e13ef66fc8b1 in_seq=0 out_seq=0 2024-05-08T22:03:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.057+0000 7f7a12ffd700 1 -- 172.21.15.64:0/3969011378 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a401ea2e60 con 0x55a401fd78a0 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.057+0000 7f7a12ffd700 10 client.25613 handle_command_reply: tid=0 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.057+0000 7f7a45da5b80 1 client.25613 shutdown 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 msgr2=0x55a401f7aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f7aea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a28004920 tx=0x7f7a28005e50 comp rx=0 tx=0).stop 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 shutdown_connections 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a401f6c2e0 0x55a401f7aea0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a401fd78a0 0x55a401fd9cc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a401f6aa90 0x55a401f79950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 --2- 172.21.15.64:0/3969011378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a401f6b3f0 0x55a401f79e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a12ffd700 0 client.25613 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 >> 172.21.15.64:0/3969011378 conn(0x55a401eb81b0 msgr2=0x55a401f56020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 shutdown_connections 2024-05-08T22:03:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 1 -- 172.21.15.64:0/3969011378 wait complete. 2024-05-08T22:03:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a45da5b80 20 client.25613 trim_cache size 0 max 16384 2024-05-08T22:03:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:03:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:03:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:03:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7a1c0a1320 msgr2=0x7f7a1c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a39beb700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7a1c0a1320 0x7f7a1c0a37e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55a401f6bac0 tx=0x7f7a28007b70 comp rx=0 tx=0).stop 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 msgr2=0x7f7a341a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.058+0000 7f7a39beb700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 0x7f7a341a6330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a240036f0 tx=0x7f7a24003d80 comp rx=0 tx=0).stop 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 shutdown_connections 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7a1c0a1320 0x7f7a1c0a37e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3410d2e0 0x7f7a341ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a34107fa0 0x7f7a341a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 --2- 172.21.15.64:0/527660325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3410a910 0x7f7a341a6330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 >> 172.21.15.64:0/527660325 conn(0x7f7a3406df30 msgr2=0x7f7a34109560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 shutdown_connections 2024-05-08T22:03:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:02.059+0000 7f7a39beb700 1 -- 172.21.15.64:0/527660325 wait complete. 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:01 smithi186 ceph-mon[46619]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 33 MiB/s wr, 1.69k op/s 2024-05-08T22:03:02.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:01 smithi064 ceph-mon[42500]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 33 MiB/s wr, 1.69k op/s 2024-05-08T22:03:02.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:01 smithi160 ceph-mon[46962]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 33 MiB/s wr, 1.69k op/s 2024-05-08T22:03:04.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:03 smithi186 ceph-mon[46619]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 1.99k op/s 2024-05-08T22:03:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:03 smithi064 ceph-mon[42500]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 1.99k op/s 2024-05-08T22:03:04.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:03 smithi160 ceph-mon[46962]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 1.99k op/s 2024-05-08T22:03:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:05 smithi186 ceph-mon[46619]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 38 MiB/s wr, 2.03k op/s 2024-05-08T22:03:06.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:05 smithi064 ceph-mon[42500]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 38 MiB/s wr, 2.03k op/s 2024-05-08T22:03:06.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:05 smithi160 ceph-mon[46962]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 38 MiB/s wr, 2.03k op/s 2024-05-08T22:03:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 495.0 s, 1974.2 tps, lat 50.898 ms stddev 67.270 2024-05-08T22:03:08.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:07 smithi186 ceph-mon[46619]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2024-05-08T22:03:08.218 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:07 smithi064 ceph-mon[42500]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2024-05-08T22:03:08.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:07 smithi160 ceph-mon[46962]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2024-05-08T22:03:10.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:09 smithi186 ceph-mon[46619]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 1.97k op/s 2024-05-08T22:03:10.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:09 smithi064 ceph-mon[42500]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 1.97k op/s 2024-05-08T22:03:10.298 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:09 smithi160 ceph-mon[46962]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 1.97k op/s 2024-05-08T22:03:11.479 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 500.0 s, 1930.0 tps, lat 51.256 ms stddev 60.021 2024-05-08T22:03:12.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:11 smithi186 ceph-mon[46619]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 1.56k op/s 2024-05-08T22:03:12.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:11 smithi064 ceph-mon[42500]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 1.56k op/s 2024-05-08T22:03:12.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:11 smithi160 ceph-mon[46962]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 1.56k op/s 2024-05-08T22:03:14.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:13 smithi064 ceph-mon[42500]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-05-08T22:03:14.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:13 smithi160 ceph-mon[46962]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-05-08T22:03:14.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:13 smithi186 ceph-mon[46619]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-05-08T22:03:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.212+0000 7fa8d93a7700 1 -- 172.21.15.64:0/2147803161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 msgr2=0x7fa8d41005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.212+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/2147803161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d41005d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009990 tx=0x7fa8c8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 -- 172.21.15.64:0/2147803161 shutdown_connections 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/2147803161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8d4100b10 0x7fa8d4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/2147803161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 0x7fa8d40ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/2147803161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d41005d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 -- 172.21.15.64:0/2147803161 >> 172.21.15.64:0/2147803161 conn(0x7fa8d40fb390 msgr2=0x7fa8d40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 -- 172.21.15.64:0/2147803161 shutdown_connections 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 -- 172.21.15.64:0/2147803161 wait complete. 2024-05-08T22:03:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.213+0000 7fa8d93a7700 1 Processor -- start 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 -- start start 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 0x7fa8d410ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d4110450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8d4100b10 0x7fa8d41ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41154f0 con 0x7fa8d4100150 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4115660 con 0x7fa8d4100b10 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d93a7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41adab0 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d4110450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d4110450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41546/0 (socket says 172.21.15.64:41546) 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d27fc700 1 -- 172.21.15.64:0/1981832357 learned_addr learned my addr 172.21.15.64:0/1981832357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 0x7fa8d410ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8d4100b10 0x7fa8d41ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/1981832357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8d4100b10 msgr2=0x7fa8d41ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8d4100b10 0x7fa8d41ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 msgr2=0x7fa8d4110450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.214+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d4110450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/1981832357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4009520 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 0x7fa8d410ff10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e9f0 tx=0x7fa8c400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8bbfff700 1 -- 172.21.15.64:0/1981832357 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400cb70 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8bbfff700 1 -- 172.21.15.64:0/1981832357 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c40044e0 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c80095f0 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8bbfff700 1 -- 172.21.15.64:0/1981832357 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003d50 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.215+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d41ae060 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.218+0000 7fa8bbfff700 1 -- 172.21.15.64:0/1981832357 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa8c4018430 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.219+0000 7fa8bbfff700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa8bc0a1270 0x7fa8bc0a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.219+0000 7fa8bbfff700 1 -- 172.21.15.64:0/1981832357 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8c4014070 con 0x7fa8d40ff7f0 2024-05-08T22:03:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.219+0000 7fa8e5561b80 1 Processor -- start 2024-05-08T22:03:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.219+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa8bc0a1270 0x7fa8bc0a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.219+0000 7fa8e5561b80 1 -- start start 2024-05-08T22:03:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 0x563989837ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989838cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989839750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398978f0f0 con 0x5639898383e0 2024-05-08T22:03:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398978cab0 con 0x563989837a80 2024-05-08T22:03:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639897883a0 con 0x5639898392d0 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8d2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 0x563989837ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8d2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 0x563989837ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:49090/0 (socket says 172.21.15.64:49090) 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/3860741513 learned_addr learned my addr 172.21.15.64:0/3860741513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.220+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989839750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.221+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/3860741513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 msgr2=0x563989839750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.221+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989839750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.221+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/3860741513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 msgr2=0x563989838cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.221+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989838cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.221+0000 7fa8d2ffd700 1 -- 172.21.15.64:0/3860741513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563989609350 con 0x563989837a80 2024-05-08T22:03:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.224+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 0x563989837ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40965f0 tx=0x7fa8c4099f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c71cd9db53a405 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.224+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa8bc0a1270 0x7fa8bc0a3730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8006010 tx=0x7fa8c80050c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.224+0000 7fa8b97fa700 1 -- 172.21.15.64:0/3860741513 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c409a7f0 con 0x563989837a80 2024-05-08T22:03:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.224+0000 7fa8b97fa700 1 -- 172.21.15.64:0/3860741513 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c409a0f0 con 0x563989837a80 2024-05-08T22:03:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.224+0000 7fa8b97fa700 1 -- 172.21.15.64:0/3860741513 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4098490 con 0x563989837a80 2024-05-08T22:03:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.225+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3860741513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 msgr2=0x563989837ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.225+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 0x563989837ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40965f0 tx=0x7fa8c4099f90 comp rx=0 tx=0).stop 2024-05-08T22:03:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3860741513 shutdown_connections 2024-05-08T22:03:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563989837a80 0x563989837ea0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989839750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3860741513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989838cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3860741513 >> 172.21.15.64:0/3860741513 conn(0x5639897961b0 msgr2=0x563989834440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3860741513 shutdown_connections 2024-05-08T22:03:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3860741513 wait complete. 2024-05-08T22:03:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.226+0000 7fa8e5561b80 1 -- start start 2024-05-08T22:03:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 Processor -- start 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989841e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989848e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989724500 con 0x5639898383e0 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639897241a0 con 0x5639898392d0 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398960b640 con 0x563989837a80 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8e5561b80 10 client.0 _ll_register_callbacks cb 0x56398972dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8d27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989841e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8d27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989841e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41564/0 (socket says 172.21.15.64:41564) 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8d27fc700 1 -- 172.21.15.64:0/3334758233 learned_addr learned my addr 172.21.15.64:0/3334758233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989848e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.227+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d27fc700 1 -- 172.21.15.64:0/3334758233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 msgr2=0x5639898418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d27fc700 1 -- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 msgr2=0x563989848e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989848e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d27fc700 1 -- 172.21.15.64:0/3334758233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d41ae060 con 0x5639898383e0 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989841e00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800bf50 tx=0x7fa8c8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82470219fed3e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/3334758233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8004c70 con 0x5639898383e0 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639896e1210 con 0x5639898383e0 2024-05-08T22:03:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.228+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/3334758233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8c8004dd0 con 0x5639898383e0 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/3334758233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8c800f460 con 0x5639898383e0 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 10 client.15858 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5639895b5b00 con 0x5639898383e0 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/3334758233 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639896e1210 con 0x5639898383e0 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/3334758233 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa8c803a070 con 0x5639898383e0 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 20 client.15858 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56398984d750 0x56398984fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 4 client.15858 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.229+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639898077b0 con 0x56398984d750 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.230+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56398984d750 0x56398984fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.231+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56398984d750 0x56398984fb70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9269d5ea7b113703 server_cookie=f9bc3f5b04d9b39b in_seq=0 out_seq=0 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.231+0000 7fa8b6ff5700 10 client.15858 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.231+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/3334758233 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639898077b0 con 0x56398984d750 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.231+0000 7fa8b6ff5700 10 client.15858 handle_command_reply: tid=0 2024-05-08T22:03:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.231+0000 7fa8e5561b80 1 client.15858 shutdown 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 msgr2=0x563989841e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989841e00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800bf50 tx=0x7fa8c8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 shutdown_connections 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989848e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56398984d750 0x56398984fb70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/3334758233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x563989841e00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8b6ff5700 0 client.15858 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.232+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 >> 172.21.15.64:0/3334758233 conn(0x5639897961b0 msgr2=0x563989832860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.233+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 shutdown_connections 2024-05-08T22:03:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.233+0000 7fa8e5561b80 1 -- 172.21.15.64:0/3334758233 wait complete. 2024-05-08T22:03:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.233+0000 7fa8e5561b80 20 client.15858 trim_cache size 0 max 16384 2024-05-08T22:03:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.270+0000 7fa8e5561b80 1 Processor -- start 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.270+0000 7fa8e5561b80 1 -- start start 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563989837a80 0x5639898494f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 0x563989848e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989840dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989838b80 con 0x563989837a80 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989839150 con 0x5639898383e0 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989841430 con 0x5639898392d0 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 0x563989848e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 0x563989848e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:49094/0 (socket says 172.21.15.64:49094) 2024-05-08T22:03:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 -- 172.21.15.64:0/1668264391 learned_addr learned my addr 172.21.15.64:0/1668264391 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989840dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 -- 172.21.15.64:0/1668264391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 msgr2=0x563989840dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989840dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 -- 172.21.15.64:0/1668264391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563989837a80 msgr2=0x5639898494f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563989837a80 0x5639898494f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.271+0000 7fa8d27fc700 1 -- 172.21.15.64:0/1668264391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56398974e500 con 0x5639898383e0 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.272+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989840dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.272+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 0x563989848e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800bf20 tx=0x7fa8c80094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7836602292eb3506 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.272+0000 7fa8b9ffb700 1 -- 172.21.15.64:0/1668264391 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8027a80 con 0x5639898383e0 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.272+0000 7fa8b9ffb700 1 -- 172.21.15.64:0/1668264391 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c8037070 con 0x5639898383e0 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.272+0000 7fa8b9ffb700 1 -- 172.21.15.64:0/1668264391 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800f870 con 0x5639898383e0 2024-05-08T22:03:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1668264391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 msgr2=0x563989848e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 0x563989848e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800bf20 tx=0x7fa8c80094b0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1668264391 shutdown_connections 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639898392d0 0x563989840dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563989837a80 0x5639898494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1668264391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898383e0 0x563989848e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1668264391 >> 172.21.15.64:0/1668264391 conn(0x5639897961b0 msgr2=0x56398983f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1668264391 shutdown_connections 2024-05-08T22:03:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1668264391 wait complete. 2024-05-08T22:03:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 -- start start 2024-05-08T22:03:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.273+0000 7fa8e5561b80 1 Processor -- start 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898461b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x5639898466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989847660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989847ba0 con 0x5639898383e0 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989847d10 con 0x5639898392d0 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563989847ed0 con 0x563989837a80 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8e5561b80 10 client.0 _ll_register_callbacks cb 0x56398972dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989847660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989847660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:49102/0 (socket says 172.21.15.64:49102) 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8d37fe700 1 -- 172.21.15.64:0/1911323325 learned_addr learned my addr 172.21.15.64:0/1911323325 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8d27fc700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x5639898466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.274+0000 7fa8d37fe700 1 -- 172.21.15.64:0/1911323325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 msgr2=0x5639898461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8d37fe700 1 -- 172.21.15.64:0/1911323325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 msgr2=0x5639898466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x5639898466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8d37fe700 1 -- 172.21.15.64:0/1911323325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639895b5b00 con 0x5639898392d0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8d37fe700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989847660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc008e50 tx=0x7fa8cc008e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4befa493147e1ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/1911323325 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00e040 con 0x5639898392d0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/1911323325 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8cc003e10 con 0x5639898392d0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8c409a0f0 con 0x5639898392d0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/1911323325 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8cc009410 con 0x5639898392d0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8e5561b80 10 client.25631 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.275+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56398960dc70 con 0x5639898392d0 2024-05-08T22:03:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/1911323325 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c409a0f0 con 0x5639898392d0 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/1911323325 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa8cc004680 con 0x5639898392d0 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8e5561b80 10 client.25631 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8e5561b80 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8e5561b80 20 client.25631 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5639898a42a0 0x5639898a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8e5561b80 4 client.25631 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563989780e60 con 0x5639898a42a0 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.276+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5639898a42a0 0x5639898a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.277+0000 7fa8d2ffd700 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5639898a42a0 0x5639898a66c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d2b8a60567096a server_cookie=a5d59ab03de97e2b in_seq=0 out_seq=0 2024-05-08T22:03:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.277+0000 7fa8b6ff5700 10 client.25631 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8b6ff5700 1 -- 172.21.15.64:0/1911323325 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563989780e60 con 0x5639898a42a0 2024-05-08T22:03:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8b6ff5700 10 client.25631 handle_command_reply: tid=0 2024-05-08T22:03:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 client.25631 shutdown 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 msgr2=0x563989847660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989847660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc008e50 tx=0x7fa8cc008e80 comp rx=0 tx=0).stop 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 shutdown_connections 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639898392d0 0x563989847660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5639898a42a0 0x5639898a66c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563989837a80 0x5639898461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 --2- 172.21.15.64:0/1911323325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639898383e0 0x5639898466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.285+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 >> 172.21.15.64:0/1911323325 conn(0x5639897961b0 msgr2=0x56398983f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8b6ff5700 0 client.25631 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 shutdown_connections 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8e5561b80 1 -- 172.21.15.64:0/1911323325 wait complete. 2024-05-08T22:03:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8e5561b80 20 client.25631 trim_cache size 0 max 16384 2024-05-08T22:03:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:03:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:03:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:03:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa8bc0a1270 msgr2=0x7fa8bc0a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa8bc0a1270 0x7fa8bc0a3730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8006010 tx=0x7fa8c80050c0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 msgr2=0x7fa8d410ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 0x7fa8d410ff10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e9f0 tx=0x7fa8c400ed00 comp rx=0 tx=0).stop 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 shutdown_connections 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa8bc0a1270 0x7fa8bc0a3730 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8d4100b10 0x7fa8d41ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d40ff7f0 0x7fa8d410ff10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 --2- 172.21.15.64:0/1981832357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d4100150 0x7fa8d4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.286+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 >> 172.21.15.64:0/1981832357 conn(0x7fa8d40fb390 msgr2=0x7fa8d40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.287+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 shutdown_connections 2024-05-08T22:03:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:15.287+0000 7fa8d93a7700 1 -- 172.21.15.64:0/1981832357 wait complete. 2024-05-08T22:03:16.212 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:15 smithi064 ceph-mon[42500]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 729 op/s 2024-05-08T22:03:16.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:15 smithi160 ceph-mon[46962]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 729 op/s 2024-05-08T22:03:16.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:15 smithi186 ceph-mon[46619]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 729 op/s 2024-05-08T22:03:16.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 505.0 s, 2234.5 tps, lat 45.181 ms stddev 54.217 2024-05-08T22:03:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:17 smithi064 ceph-mon[42500]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 733 op/s 2024-05-08T22:03:18.299 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:17 smithi160 ceph-mon[46962]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 733 op/s 2024-05-08T22:03:18.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:17 smithi186 ceph-mon[46619]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 733 op/s 2024-05-08T22:03:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:19 smithi064 ceph-mon[42500]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 673 op/s 2024-05-08T22:03:20.300 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:19 smithi160 ceph-mon[46962]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 673 op/s 2024-05-08T22:03:20.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:19 smithi186 ceph-mon[46619]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 673 op/s 2024-05-08T22:03:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 510.0 s, 1984.8 tps, lat 50.489 ms stddev 57.041 2024-05-08T22:03:22.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:21 smithi160 ceph-mon[46962]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 715 op/s 2024-05-08T22:03:22.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:21 smithi186 ceph-mon[46619]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 715 op/s 2024-05-08T22:03:22.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:21 smithi064 ceph-mon[42500]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 715 op/s 2024-05-08T22:03:24.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:23 smithi160 ceph-mon[46962]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 664 op/s 2024-05-08T22:03:24.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:23 smithi186 ceph-mon[46619]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 664 op/s 2024-05-08T22:03:24.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:23 smithi064 ceph-mon[42500]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 664 op/s 2024-05-08T22:03:26.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:26 smithi160 ceph-mon[46962]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 678 op/s 2024-05-08T22:03:26.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:26 smithi186 ceph-mon[46619]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 678 op/s 2024-05-08T22:03:26.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:26 smithi064 ceph-mon[42500]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 678 op/s 2024-05-08T22:03:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 515.0 s, 1848.4 tps, lat 54.209 ms stddev 61.872 2024-05-08T22:03:28.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:28 smithi160 ceph-mon[46962]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 653 op/s 2024-05-08T22:03:28.397 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:28 smithi186 ceph-mon[46619]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 653 op/s 2024-05-08T22:03:28.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:28 smithi064 ceph-mon[42500]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 653 op/s 2024-05-08T22:03:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 -- 172.21.15.64:0/1984199966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2748100800 msgr2=0x7f27481050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 --2- 172.21.15.64:0/1984199966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2748100800 0x7f27481050a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f274000b0a0 tx=0x7f274000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 -- 172.21.15.64:0/1984199966 shutdown_connections 2024-05-08T22:03:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 --2- 172.21.15.64:0/1984199966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2748100800 0x7f27481050a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 --2- 172.21.15.64:0/1984199966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27480ff0e0 0x7f27480ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 --2- 172.21.15.64:0/1984199966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f27481002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.473+0000 7f274c87f700 1 -- 172.21.15.64:0/1984199966 >> 172.21.15.64:0/1984199966 conn(0x7f27480faa30 msgr2=0x7f27480fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.474+0000 7f274c87f700 1 -- 172.21.15.64:0/1984199966 shutdown_connections 2024-05-08T22:03:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.474+0000 7f274c87f700 1 -- 172.21.15.64:0/1984199966 wait complete. 2024-05-08T22:03:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.474+0000 7f274c87f700 1 Processor -- start 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.478+0000 7f274c87f700 1 -- start start 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f274c87f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27480ff0e0 0x7f274806f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f274c87f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f274806fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f274c87f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2748100800 0x7f27481ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f274c87f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748070440 con 0x7f27480ff0e0 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f274c87f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480705b0 con 0x7f27480ffe40 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f274c87f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27481ada30 con 0x7f2748100800 2024-05-08T22:03:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f274806fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f274806fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60914/0 (socket says 172.21.15.64:60914) 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2746d9d700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2748100800 0x7f27481ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 -- 172.21.15.64:0/630513283 learned_addr learned my addr 172.21.15.64:0/630513283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 -- 172.21.15.64:0/630513283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2748100800 msgr2=0x7f27481ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2748100800 0x7f27481ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 -- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27480ff0e0 msgr2=0x7f274806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.479+0000 7f2745d9b700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27480ff0e0 0x7f274806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.480+0000 7f2745d9b700 1 -- 172.21.15.64:0/630513283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2740009d00 con 0x7f27480ffe40 2024-05-08T22:03:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.480+0000 7f2745d9b700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f274806fe70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c920 tx=0x7f273c00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.480+0000 7f27377fe700 1 -- 172.21.15.64:0/630513283 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c007b00 con 0x7f27480ffe40 2024-05-08T22:03:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.480+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27481add10 con 0x7f27480ffe40 2024-05-08T22:03:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.480+0000 7f27377fe700 1 -- 172.21.15.64:0/630513283 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f273c00ce70 con 0x7f27480ffe40 2024-05-08T22:03:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.480+0000 7f27377fe700 1 -- 172.21.15.64:0/630513283 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c005460 con 0x7f27480ffe40 2024-05-08T22:03:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.481+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27481ae210 con 0x7f27480ffe40 2024-05-08T22:03:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.484+0000 7f27377fe700 1 -- 172.21.15.64:0/630513283 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f273c03e4e0 con 0x7f27480ffe40 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.485+0000 7f2758a39b80 1 Processor -- start 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.485+0000 7f27377fe700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27300a1210 0x7f27300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.485+0000 7f27377fe700 1 -- 172.21.15.64:0/630513283 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f273c0cbbb0 con 0x7f27480ffe40 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 -- start start 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dc3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dc4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 0x561be9dc5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9d1b0f0 con 0x561be9dc52d0 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9d18ab0 con 0x561be9dc43e0 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2758a39b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9d143a0 con 0x561be9dc3a80 2024-05-08T22:03:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2746d9d700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27300a1210 0x7f27300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dc4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dc4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60916/0 (socket says 172.21.15.64:60916) 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.486+0000 7f2745d9b700 1 -- 172.21.15.64:0/2747679032 learned_addr learned my addr 172.21.15.64:0/2747679032 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f274659c700 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dc3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2745d9b700 1 -- 172.21.15.64:0/2747679032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 msgr2=0x561be9dc3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dc3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2745d9b700 1 -- 172.21.15.64:0/2747679032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 msgr2=0x561be9dc5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 0x561be9dc5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2745d9b700 1 -- 172.21.15.64:0/2747679032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561be9b95350 con 0x561be9dc43e0 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dc4cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f273c0a0f00 tx=0x7f273c098bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2be6aa25bbb56d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.487+0000 7f2734ff9700 1 -- 172.21.15.64:0/2747679032 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c09cde0 con 0x561be9dc43e0 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.488+0000 7f2734ff9700 1 -- 172.21.15.64:0/2747679032 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f273c0a1430 con 0x561be9dc43e0 2024-05-08T22:03:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.488+0000 7f2734ff9700 1 -- 172.21.15.64:0/2747679032 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c0a1200 con 0x561be9dc43e0 2024-05-08T22:03:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.488+0000 7f2758a39b80 1 -- 172.21.15.64:0/2747679032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 msgr2=0x561be9dc4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.488+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dc4cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f273c0a0f00 tx=0x7f273c098bd0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 -- 172.21.15.64:0/2747679032 shutdown_connections 2024-05-08T22:03:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 0x561be9dc5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dc3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2747679032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dc4cf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 -- 172.21.15.64:0/2747679032 >> 172.21.15.64:0/2747679032 conn(0x561be9d221b0 msgr2=0x561be9dc0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2746d9d700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27300a1210 0x7f27300a36d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x561be9dc4ab0 tx=0x7f2740009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 -- 172.21.15.64:0/2747679032 shutdown_connections 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.490+0000 7f2758a39b80 1 -- 172.21.15.64:0/2747679032 wait complete. 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 -- start start 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 Processor -- start 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 0x561be9dcde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9cb0500 con 0x561be9dc43e0 2024-05-08T22:03:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9cb01a0 con 0x561be9dc52d0 2024-05-08T22:03:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9b97640 con 0x561be9dc3a80 2024-05-08T22:03:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2758a39b80 10 client.0 _ll_register_callbacks cb 0x561be9cb9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 0x561be9dcde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.491+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 0x561be9dcde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60776/0 (socket says 172.21.15.64:60776) 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 -- 172.21.15.64:0/2239641543 learned_addr learned my addr 172.21.15.64:0/2239641543 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f274659c700 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 -- 172.21.15.64:0/2239641543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 msgr2=0x561be9dcd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 -- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 msgr2=0x561be9dd4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 -- 172.21.15.64:0/2239641543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27481ae210 con 0x561be9dc43e0 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 0x561be9dcde70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f273c000c00 tx=0x7f273c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8656df2d49529dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.492+0000 7f272dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.493+0000 7f272dffb700 1 -- 172.21.15.64:0/2239641543 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c09cde0 con 0x561be9dc43e0 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.493+0000 7f272dffb700 1 -- 172.21.15.64:0/2239641543 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f273c0b7030 con 0x561be9dc43e0 2024-05-08T22:03:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.493+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561be9c6d210 con 0x561be9dc43e0 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.493+0000 7f272dffb700 1 -- 172.21.15.64:0/2239641543 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f273c0b1b60 con 0x561be9dc43e0 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.493+0000 7f2758a39b80 10 client.15867 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.493+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561be9b41b00 con 0x561be9dc43e0 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f272dffb700 1 -- 172.21.15.64:0/2239641543 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561be9c6d210 con 0x561be9dc43e0 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f272dffb700 1 -- 172.21.15.64:0/2239641543 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f273c097ff0 con 0x561be9dc43e0 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f2758a39b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f2758a39b80 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f2758a39b80 20 client.15867 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9dd9430 0x561be9ddb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f2758a39b80 4 client.15867 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561be9d937b0 con 0x561be9dd9430 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.494+0000 7f274659c700 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9dd9430 0x561be9ddb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.495+0000 7f272dffb700 10 client.15867 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.495+0000 7f274659c700 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9dd9430 0x561be9ddb850 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3f23ca0594ba8 server_cookie=e9fb3de833515f34 in_seq=0 out_seq=0 2024-05-08T22:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.495+0000 7f272dffb700 1 -- 172.21.15.64:0/2239641543 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561be9d937b0 con 0x561be9dd9430 2024-05-08T22:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.495+0000 7f272dffb700 10 client.15867 handle_command_reply: tid=0 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.495+0000 7f2758a39b80 1 client.15867 shutdown 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.495+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 msgr2=0x561be9dcde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 0x561be9dcde70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f273c000c00 tx=0x7f273c00a650 comp rx=0 tx=0).stop 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 shutdown_connections 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9dd9430 0x561be9ddb850 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2239641543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc43e0 0x561be9dcde70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 >> 172.21.15.64:0/2239641543 conn(0x561be9d221b0 msgr2=0x561be9dbe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f272dffb700 0 client.15867 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 shutdown_connections 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 1 -- 172.21.15.64:0/2239641543 wait complete. 2024-05-08T22:03:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.496+0000 7f2758a39b80 20 client.15867 trim_cache size 0 max 16384 2024-05-08T22:03:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.518+0000 7f2758a39b80 1 Processor -- start 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.518+0000 7f2758a39b80 1 -- start start 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.518+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.518+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dd4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.518+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 0x561be9dc56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.518+0000 7f2758a39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9db97c0 con 0x561be9dc52d0 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2758a39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9dbb9b0 con 0x561be9dc43e0 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2758a39b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9dce830 con 0x561be9dc3a80 2024-05-08T22:03:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dd4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dd4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:60928/0 (socket says 172.21.15.64:60928) 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 -- 172.21.15.64:0/2981218019 learned_addr learned my addr 172.21.15.64:0/2981218019 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f274659c700 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 -- 172.21.15.64:0/2981218019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 msgr2=0x561be9dcce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 -- 172.21.15.64:0/2981218019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 msgr2=0x561be9dc56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 0x561be9dc56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 -- 172.21.15.64:0/2981218019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561be9cda500 con 0x561be9dc43e0 2024-05-08T22:03:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.519+0000 7f2745d9b700 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dd4e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f273c095590 tx=0x7f273c004e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e792ce108fe5c1f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.520+0000 7f27357fa700 1 -- 172.21.15.64:0/2981218019 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c098740 con 0x561be9dc43e0 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.520+0000 7f27357fa700 1 -- 172.21.15.64:0/2981218019 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f273c09cde0 con 0x561be9dc43e0 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.520+0000 7f27357fa700 1 -- 172.21.15.64:0/2981218019 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c09e100 con 0x561be9dc43e0 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 -- 172.21.15.64:0/2981218019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 msgr2=0x561be9dd4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dd4e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f273c095590 tx=0x7f273c004e60 comp rx=0 tx=0).stop 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 -- 172.21.15.64:0/2981218019 shutdown_connections 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc52d0 0x561be9dc56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc3a80 0x561be9dcce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 --2- 172.21.15.64:0/2981218019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc43e0 0x561be9dd4e70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 -- 172.21.15.64:0/2981218019 >> 172.21.15.64:0/2981218019 conn(0x561be9d221b0 msgr2=0x561be9dcb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 -- 172.21.15.64:0/2981218019 shutdown_connections 2024-05-08T22:03:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.521+0000 7f2758a39b80 1 -- 172.21.15.64:0/2981218019 wait complete. 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 -- start start 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 Processor -- start 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc3a80 0x561be9dd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc43e0 0x561be9dd1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9dd21e0 con 0x561be9dc3a80 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9dd2350 con 0x561be9dc52d0 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be9e2d8d0 con 0x561be9dc43e0 2024-05-08T22:03:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.522+0000 7f2758a39b80 10 client.0 _ll_register_callbacks cb 0x561be9cb9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.523+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc43e0 0x561be9dd1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.523+0000 7f2745d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc43e0 0x561be9dd1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:60688/0 (socket says 172.21.15.64:60688) 2024-05-08T22:03:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.523+0000 7f2745d9b700 1 -- 172.21.15.64:0/3249825068 learned_addr learned my addr 172.21.15.64:0/3249825068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 -- 172.21.15.64:0/3249825068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc43e0 msgr2=0x561be9dd1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc43e0 0x561be9dd1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 -- 172.21.15.64:0/3249825068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc3a80 msgr2=0x561be9dd4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc3a80 0x561be9dd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 -- 172.21.15.64:0/3249825068 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561be9b41b00 con 0x561be9dc52d0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f2746d9d700 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd1ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2740009fd0 tx=0x7f274000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86a4797515e54b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f272dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.524+0000 7f272dffb700 1 -- 172.21.15.64:0/3249825068 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000e040 con 0x561be9dc52d0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f272dffb700 1 -- 172.21.15.64:0/3249825068 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2740005040 con 0x561be9dc52d0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f273c0a1430 con 0x561be9dc52d0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f272dffb700 1 -- 172.21.15.64:0/3249825068 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2740007840 con 0x561be9dc52d0 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 10 client.25643 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561be9b99c70 con 0x561be9dc52d0 2024-05-08T22:03:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f272dffb700 1 -- 172.21.15.64:0/3249825068 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f273c0a1430 con 0x561be9dc52d0 2024-05-08T22:03:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f272dffb700 1 -- 172.21.15.64:0/3249825068 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f27400266e0 con 0x561be9dc52d0 2024-05-08T22:03:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 10 client.25643 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 20 client.25643 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9e2f3f0 0x561be9e31810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 4 client.25643 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.525+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561be9d0ce60 con 0x561be9e2f3f0 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.526+0000 7f274659c700 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9e2f3f0 0x561be9e31810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.526+0000 7f274659c700 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9e2f3f0 0x561be9e31810 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8edbfe155e9ed0cc server_cookie=e925bfe6b6202d6a in_seq=0 out_seq=0 2024-05-08T22:03:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.526+0000 7f272dffb700 10 client.25643 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.534+0000 7f272dffb700 1 -- 172.21.15.64:0/3249825068 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561be9d0ce60 con 0x561be9e2f3f0 2024-05-08T22:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.534+0000 7f272dffb700 10 client.25643 handle_command_reply: tid=0 2024-05-08T22:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.534+0000 7f2758a39b80 1 client.25643 shutdown 2024-05-08T22:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.534+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 msgr2=0x561be9dd1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.534+0000 7f2758a39b80 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd1ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2740009fd0 tx=0x7f274000bd30 comp rx=0 tx=0).stop 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 shutdown_connections 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f2758a39b80 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561be9dc52d0 0x561be9dd1ca0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f2758a39b80 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561be9e2f3f0 0x561be9e31810 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f2758a39b80 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561be9dc3a80 0x561be9dd4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f2758a39b80 1 --2- 172.21.15.64:0/3249825068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561be9dc43e0 0x561be9dd1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 >> 172.21.15.64:0/3249825068 conn(0x561be9d221b0 msgr2=0x561be9dc0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.535+0000 7f272dffb700 0 client.25643 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.536+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 shutdown_connections 2024-05-08T22:03:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.537+0000 7f2758a39b80 1 -- 172.21.15.64:0/3249825068 wait complete. 2024-05-08T22:03:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.538+0000 7f2758a39b80 20 client.25643 trim_cache size 0 max 16384 2024-05-08T22:03:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:03:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:03:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:03:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27300a1210 msgr2=0x7f27300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27300a1210 0x7f27300a36d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x561be9dc4ab0 tx=0x7f2740009c00 comp rx=0 tx=0).stop 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 msgr2=0x7f274806fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f274806fe70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c920 tx=0x7f273c00cc30 comp rx=0 tx=0).stop 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 shutdown_connections 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f27300a1210 0x7f27300a36d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2748100800 0x7f27481ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27480ff0e0 0x7f274806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 --2- 172.21.15.64:0/630513283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27480ffe40 0x7f274806fe70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.539+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 >> 172.21.15.64:0/630513283 conn(0x7f27480faa30 msgr2=0x7f2748101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.541+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 shutdown_connections 2024-05-08T22:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:28.541+0000 7f274c87f700 1 -- 172.21.15.64:0/630513283 wait complete. 2024-05-08T22:03:30.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:30 smithi160 ceph-mon[46962]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 644 op/s 2024-05-08T22:03:30.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:30 smithi186 ceph-mon[46619]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 644 op/s 2024-05-08T22:03:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:30 smithi064 ceph-mon[42500]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 644 op/s 2024-05-08T22:03:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 520.0 s, 2563.8 tps, lat 39.117 ms stddev 47.647 2024-05-08T22:03:32.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:32 smithi160 ceph-mon[46962]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-05-08T22:03:32.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:32 smithi186 ceph-mon[46619]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-05-08T22:03:32.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:32 smithi064 ceph-mon[42500]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 761 op/s 2024-05-08T22:03:33.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:33 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:33.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:33 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:33.643 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:33 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:34.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 707 op/s 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:34.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:03:34.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 707 op/s 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:34.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:03:34.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 707 op/s 2024-05-08T22:03:34.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:34.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:34.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:34.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:03:35.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:35 smithi064 ceph-mon[42500]: Adjusting osd_memory_target on smithi186 to 182.5M 2024-05-08T22:03:35.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:35 smithi064 ceph-mon[42500]: Unable to set osd_memory_target on smithi186 to 191373414: error parsing value: Value '191373414' is below minimum 939524096 2024-05-08T22:03:35.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:35 smithi160 ceph-mon[46962]: Adjusting osd_memory_target on smithi186 to 182.5M 2024-05-08T22:03:35.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:35 smithi160 ceph-mon[46962]: Unable to set osd_memory_target on smithi186 to 191373414: error parsing value: Value '191373414' is below minimum 939524096 2024-05-08T22:03:35.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:35 smithi186 ceph-mon[46619]: Adjusting osd_memory_target on smithi186 to 182.5M 2024-05-08T22:03:35.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:35 smithi186 ceph-mon[46619]: Unable to set osd_memory_target on smithi186 to 191373414: error parsing value: Value '191373414' is below minimum 939524096 2024-05-08T22:03:36.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:36 smithi064 ceph-mon[42500]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 744 op/s 2024-05-08T22:03:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 525.0 s, 1183.4 tps, lat 82.393 ms stddev 102.991 2024-05-08T22:03:36.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:36 smithi160 ceph-mon[46962]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 744 op/s 2024-05-08T22:03:36.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:36 smithi186 ceph-mon[46619]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 744 op/s 2024-05-08T22:03:38.471 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:38 smithi064 ceph-mon[42500]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 818 op/s 2024-05-08T22:03:38.546 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:38 smithi160 ceph-mon[46962]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 818 op/s 2024-05-08T22:03:38.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:38 smithi186 ceph-mon[46619]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 818 op/s 2024-05-08T22:03:40.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:40 smithi064 ceph-mon[42500]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-05-08T22:03:40.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:40 smithi160 ceph-mon[46962]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-05-08T22:03:40.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:40 smithi186 ceph-mon[46619]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-05-08T22:03:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 530.0 s, 1496.2 tps, lat 67.994 ms stddev 71.731 2024-05-08T22:03:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 -- 172.21.15.64:0/4112883041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 msgr2=0x7f97c01079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/4112883041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c01079c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0099c0 tx=0x7f97bc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 -- 172.21.15.64:0/4112883041 shutdown_connections 2024-05-08T22:03:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/4112883041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c00ff580 0x7f97c00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/4112883041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c01079c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/4112883041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c0100770 0x7f97c0100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 -- 172.21.15.64:0/4112883041 >> 172.21.15.64:0/4112883041 conn(0x7f97c00faaf0 msgr2=0x7f97c00fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 -- 172.21.15.64:0/4112883041 shutdown_connections 2024-05-08T22:03:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.689+0000 7f97c7ed0700 1 -- 172.21.15.64:0/4112883041 wait complete. 2024-05-08T22:03:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.690+0000 7f97c7ed0700 1 Processor -- start 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.694+0000 7f97c7ed0700 1 -- start start 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.694+0000 7f97c7ed0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 0x7f97c010c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c7ed0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0100770 0x7f97c019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c7ed0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c7ed0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019e880 con 0x7f97c0101110 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c7ed0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019e9f0 con 0x7f97c0100770 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c7ed0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019ebb0 con 0x7f97c00ff580 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 0x7f97c010c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c646d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c546b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0100770 0x7f97c019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 0x7f97c010c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:36834/0 (socket says 172.21.15.64:36834) 2024-05-08T22:03:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 -- 172.21.15.64:0/2792696135 learned_addr learned my addr 172.21.15.64:0/2792696135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 -- 172.21.15.64:0/2792696135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0100770 msgr2=0x7f97c019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0100770 0x7f97c019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 -- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 msgr2=0x7f97c019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 -- 172.21.15.64:0/2792696135 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0095f0 con 0x7f97c00ff580 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c646d700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 0x7f97c010c470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97b800efc0 tx=0x7f97b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.695+0000 7f97b6ffd700 1 -- 172.21.15.64:0/2792696135 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8018070 con 0x7f97c00ff580 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.696+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c019ee30 con 0x7f97c00ff580 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.696+0000 7f97b6ffd700 1 -- 172.21.15.64:0/2792696135 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b800f040 con 0x7f97c00ff580 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.696+0000 7f97b6ffd700 1 -- 172.21.15.64:0/2792696135 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004280 con 0x7f97c00ff580 2024-05-08T22:03:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.696+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c019f260 con 0x7f97c00ff580 2024-05-08T22:03:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.698+0000 7f97b6ffd700 1 -- 172.21.15.64:0/2792696135 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f97b8003a00 con 0x7f97c00ff580 2024-05-08T22:03:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.698+0000 7f97b6ffd700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97ac0a1120 0x7f97ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.699+0000 7f97b6ffd700 1 -- 172.21.15.64:0/2792696135 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97b80cc7a0 con 0x7f97c00ff580 2024-05-08T22:03:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.699+0000 7f97c546b700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97ac0a1120 0x7f97ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.699+0000 7f97c546b700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97ac0a1120 0x7f97ac0a35e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97c010c9d0 tx=0x7f97b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.699+0000 7f97d408ab80 1 Processor -- start 2024-05-08T22:03:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.699+0000 7f97d408ab80 1 -- start start 2024-05-08T22:03:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deaf2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 0x5627deaf3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deaf4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97d408ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627dea4a0f0 con 0x5627deaf2a80 2024-05-08T22:03:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97d408ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627dea47ab0 con 0x5627deaf33e0 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97d408ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627dea433a0 con 0x5627deaf42d0 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 0x5627deaf3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c5c6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deaf2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 0x5627deaf3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:33566/0 (socket says 172.21.15.64:33566) 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 -- 172.21.15.64:0/2983510823 learned_addr learned my addr 172.21.15.64:0/2983510823 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deaf2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43440/0 (socket says 172.21.15.64:43440) 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c646d700 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deaf4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 -- 172.21.15.64:0/2983510823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 msgr2=0x5627deaf4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deaf4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 -- 172.21.15.64:0/2983510823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 msgr2=0x5627deaf2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deaf2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c546b700 1 -- 172.21.15.64:0/2983510823 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627de8c4350 con 0x5627deaf33e0 2024-05-08T22:03:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.700+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deaf2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:03:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.707+0000 7f97c546b700 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 0x5627deaf3cf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97b0011d50 tx=0x7f97b0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeeee097cdfe7b29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.724+0000 7f97abfff700 1 -- 172.21.15.64:0/2983510823 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000eb20 con 0x5627deaf33e0 2024-05-08T22:03:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.724+0000 7f97abfff700 1 -- 172.21.15.64:0/2983510823 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b0021070 con 0x5627deaf33e0 2024-05-08T22:03:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.724+0000 7f97abfff700 1 -- 172.21.15.64:0/2983510823 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b001d670 con 0x5627deaf33e0 2024-05-08T22:03:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.727+0000 7f97d408ab80 1 -- 172.21.15.64:0/2983510823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 msgr2=0x5627deaf3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.727+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 0x5627deaf3cf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97b0011d50 tx=0x7f97b0014000 comp rx=0 tx=0).stop 2024-05-08T22:03:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.728+0000 7f97d408ab80 1 -- 172.21.15.64:0/2983510823 shutdown_connections 2024-05-08T22:03:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.728+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deaf4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.728+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deaf2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.728+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2983510823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf33e0 0x5627deaf3cf0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97b0011d50 tx=0x7f97b0014000 comp rx=0 tx=0).stop 2024-05-08T22:03:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.728+0000 7f97d408ab80 1 -- 172.21.15.64:0/2983510823 >> 172.21.15.64:0/2983510823 conn(0x5627dea511b0 msgr2=0x5627deaef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.739+0000 7f97d408ab80 1 -- 172.21.15.64:0/2983510823 shutdown_connections 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.751+0000 7f97d408ab80 1 -- 172.21.15.64:0/2983510823 wait complete. 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.751+0000 7f97d408ab80 1 -- start start 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.751+0000 7f97d408ab80 1 Processor -- start 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf2a80 0x5627deaf85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaf8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deaf9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627de9df500 con 0x5627deaf42d0 2024-05-08T22:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627de9df1a0 con 0x5627deaf9af0 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627de8c6640 con 0x5627deaf2a80 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.752+0000 7f97d408ab80 10 client.0 _ll_register_callbacks cb 0x5627de9e8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.753+0000 7f97c646d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deaf9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.753+0000 7f97c646d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deaf9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:33576/0 (socket says 172.21.15.64:33576) 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.753+0000 7f97c646d700 1 -- 172.21.15.64:0/2949837674 learned_addr learned my addr 172.21.15.64:0/2949837674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.754+0000 7f97c646d700 1 -- 172.21.15.64:0/2949837674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf2a80 msgr2=0x5627deaf85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.754+0000 7f97c646d700 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf2a80 0x5627deaf85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.754+0000 7f97c646d700 1 -- 172.21.15.64:0/2949837674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 msgr2=0x5627deaf8ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.754+0000 7f97c646d700 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaf8ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.754+0000 7f97c646d700 1 -- 172.21.15.64:0/2949837674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c019f260 con 0x5627deaf9af0 2024-05-08T22:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.754+0000 7f97c646d700 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deaf9f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00b580 tx=0x7f97bc00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafb5570e962f6c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:41.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.762+0000 7f97a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:41.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.762+0000 7f97a97fa700 1 -- 172.21.15.64:0/2949837674 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc01c040 con 0x5627deaf9af0 2024-05-08T22:03:41.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.762+0000 7f97a97fa700 1 -- 172.21.15.64:0/2949837674 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97bc00fca0 con 0x5627deaf9af0 2024-05-08T22:03:41.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.762+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627de99c210 con 0x5627deaf9af0 2024-05-08T22:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.777+0000 7f97a97fa700 1 -- 172.21.15.64:0/2949837674 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97bc00fe60 con 0x5627deaf9af0 2024-05-08T22:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.777+0000 7f97d408ab80 10 client.25655 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.777+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f97b0017070 con 0x5627deaf9af0 2024-05-08T22:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97a97fa700 1 -- 172.21.15.64:0/2949837674 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627de99c210 con 0x5627deaf9af0 2024-05-08T22:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97a97fa700 1 -- 172.21.15.64:0/2949837674 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f97bc024450 con 0x5627deaf9af0 2024-05-08T22:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97d408ab80 10 client.25655 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97d408ab80 10 client.25655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97d408ab80 20 client.25655 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb08ea0 0x5627deb0b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97d408ab80 4 client.25655 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.788+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627deac27b0 con 0x5627deb08ea0 2024-05-08T22:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.789+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb08ea0 0x5627deb0b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.790+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb08ea0 0x5627deb0b2c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42531891b1c3293 server_cookie=5fd11dc9b4b26346 in_seq=0 out_seq=0 2024-05-08T22:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.790+0000 7f97a97fa700 10 client.25655 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.791+0000 7f97a97fa700 1 -- 172.21.15.64:0/2949837674 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627deac27b0 con 0x5627deb08ea0 2024-05-08T22:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.791+0000 7f97a97fa700 10 client.25655 handle_command_reply: tid=0 2024-05-08T22:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.791+0000 7f97d408ab80 1 client.25655 shutdown 2024-05-08T22:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.791+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 msgr2=0x5627deaf9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.791+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deaf9f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00b580 tx=0x7f97bc00f670 comp rx=0 tx=0).stop 2024-05-08T22:03:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.796+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 shutdown_connections 2024-05-08T22:03:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.796+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deaf9f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.796+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb08ea0 0x5627deb0b2c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.796+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf2a80 0x5627deaf85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.796+0000 7f97d408ab80 1 --2- 172.21.15.64:0/2949837674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaf8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.796+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 >> 172.21.15.64:0/2949837674 conn(0x5627dea511b0 msgr2=0x5627deaed860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.797+0000 7f97a97fa700 0 client.25655 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.797+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 shutdown_connections 2024-05-08T22:03:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.797+0000 7f97d408ab80 1 -- 172.21.15.64:0/2949837674 wait complete. 2024-05-08T22:03:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.797+0000 7f97d408ab80 20 client.25655 trim_cache size 0 max 16384 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 Processor -- start 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 -- start start 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf2a80 0x5627deaf3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf9af0 0x5627deb0fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627deb05690 con 0x5627deaf42d0 2024-05-08T22:03:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627deb06220 con 0x5627deaf2a80 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97d408ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627deaf40c0 con 0x5627deaf9af0 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97c546b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97c646d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf9af0 0x5627deb0fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97c546b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43448/0 (socket says 172.21.15.64:43448) 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.819+0000 7f97c546b700 1 -- 172.21.15.64:0/1710642818 learned_addr learned my addr 172.21.15.64:0/1710642818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf2a80 0x5627deaf3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c646d700 1 -- 172.21.15.64:0/1710642818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf2a80 msgr2=0x5627deaf3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c646d700 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf2a80 0x5627deaf3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c646d700 1 -- 172.21.15.64:0/1710642818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 msgr2=0x5627deaef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c646d700 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c646d700 1 -- 172.21.15.64:0/1710642818 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627de870b00 con 0x5627deaf9af0 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf2a80 0x5627deaf3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:03:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c546b700 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97c646d700 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf9af0 0x5627deb0fc20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97bc003760 tx=0x7f97bc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d7f4f2fec5dfb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97b4ff9700 1 -- 172.21.15.64:0/1710642818 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc01c040 con 0x5627deaf9af0 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97b4ff9700 1 -- 172.21.15.64:0/1710642818 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97bc024ad0 con 0x5627deaf9af0 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.820+0000 7f97b4ff9700 1 -- 172.21.15.64:0/1710642818 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc003a60 con 0x5627deaf9af0 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.821+0000 7f97d408ab80 1 -- 172.21.15.64:0/1710642818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf9af0 msgr2=0x5627deb0fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.821+0000 7f97d408ab80 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf9af0 0x5627deb0fc20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97bc003760 tx=0x7f97bc009310 comp rx=0 tx=0).stop 2024-05-08T22:03:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.822+0000 7f97d408ab80 1 -- 172.21.15.64:0/1710642818 shutdown_connections 2024-05-08T22:03:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.822+0000 7f97d408ab80 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf2a80 0x5627deaf3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.822+0000 7f97d408ab80 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf9af0 0x5627deb0fc20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.822+0000 7f97d408ab80 1 --2- 172.21.15.64:0/1710642818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf42d0 0x5627deaef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.822+0000 7f97d408ab80 1 -- 172.21.15.64:0/1710642818 >> 172.21.15.64:0/1710642818 conn(0x5627dea511b0 msgr2=0x5627deaed860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.822+0000 7f97d408ab80 1 -- 172.21.15.64:0/1710642818 shutdown_connections 2024-05-08T22:03:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.825+0000 7f97d408ab80 1 -- 172.21.15.64:0/1710642818 wait complete. 2024-05-08T22:03:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 -- start start 2024-05-08T22:03:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 Processor -- start 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deb03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deb03b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deb04080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627deb045c0 con 0x5627deaf2a80 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627deb04730 con 0x5627deaf9af0 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627deb048f0 con 0x5627deaf42d0 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97d408ab80 10 client.0 _ll_register_callbacks cb 0x5627de9e8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97c5c6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deb03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97c546b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deb03b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.826+0000 7f97c5c6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deb03600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43452/0 (socket says 172.21.15.64:43452) 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 -- 172.21.15.64:0/3469168616 learned_addr learned my addr 172.21.15.64:0/3469168616 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c646d700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deb04080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 -- 172.21.15.64:0/3469168616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 msgr2=0x5627deb03b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deb03b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 -- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 msgr2=0x5627deb04080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deb04080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 -- 172.21.15.64:0/3469168616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b002a040 con 0x5627deaf2a80 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c646d700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deb04080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97c5c6c700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deb03600 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f97b8096700 tx=0x7f97b800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60090a1462ee901 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97a97fa700 1 -- 172.21.15.64:0/3469168616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8099c90 con 0x5627deaf2a80 2024-05-08T22:03:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97a97fa700 1 -- 172.21.15.64:0/3469168616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97b809dac0 con 0x5627deaf2a80 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97b0021070 con 0x5627deaf2a80 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97a97fa700 1 -- 172.21.15.64:0/3469168616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b80a2080 con 0x5627deaf2a80 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97d408ab80 10 client.15882 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.827+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f97b0017070 con 0x5627deaf2a80 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97a97fa700 1 -- 172.21.15.64:0/3469168616 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b8065030 con 0x5627deaf2a80 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97a97fa700 1 -- 172.21.15.64:0/3469168616 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f97b80a1bf0 con 0x5627deaf2a80 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97d408ab80 10 client.15882 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97d408ab80 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97d408ab80 20 client.15882 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97d408ab80 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb81db0 0x5627deb841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97d408ab80 4 client.15882 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5627dea3be60 con 0x5627deb81db0 2024-05-08T22:03:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97c546b700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb81db0 0x5627deb841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97c546b700 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb81db0 0x5627deb841d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e490a45395e9f739 server_cookie=d81a6a13fbedd0b in_seq=0 out_seq=0 2024-05-08T22:03:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.828+0000 7f97a97fa700 10 client.15882 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.835+0000 7f97a97fa700 1 -- 172.21.15.64:0/3469168616 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5627dea3be60 con 0x5627deb81db0 2024-05-08T22:03:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.835+0000 7f97a97fa700 10 client.15882 handle_command_reply: tid=0 2024-05-08T22:03:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.835+0000 7f97d408ab80 1 client.15882 shutdown 2024-05-08T22:03:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.835+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 msgr2=0x5627deb03600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.835+0000 7f97d408ab80 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deb03600 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f97b8096700 tx=0x7f97b800cad0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 shutdown_connections 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5627deaf9af0 0x5627deb04080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5627deb81db0 0x5627deb841d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627deaf2a80 0x5627deb03600 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 --2- 172.21.15.64:0/3469168616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627deaf42d0 0x5627deb03b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 >> 172.21.15.64:0/3469168616 conn(0x5627dea511b0 msgr2=0x5627deb0fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97a97fa700 0 client.15882 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 shutdown_connections 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 1 -- 172.21.15.64:0/3469168616 wait complete. 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.836+0000 7f97d408ab80 20 client.15882 trim_cache size 0 max 16384 2024-05-08T22:03:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:03:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:03:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:03:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:03:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97ac0a1120 msgr2=0x7f97ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97ac0a1120 0x7f97ac0a35e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97c010c9d0 tx=0x7f97b0009260 comp rx=0 tx=0).stop 2024-05-08T22:03:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 msgr2=0x7f97c010c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 0x7f97c010c470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97b800efc0 tx=0x7f97b800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 shutdown_connections 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f97ac0a1120 0x7f97ac0a35e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97c0101110 0x7f97c019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97c00ff580 0x7f97c010c470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 --2- 172.21.15.64:0/2792696135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0100770 0x7f97c019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 >> 172.21.15.64:0/2792696135 conn(0x7f97c00faaf0 msgr2=0x7f97c00faed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 shutdown_connections 2024-05-08T22:03:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:41.837+0000 7f97c7ed0700 1 -- 172.21.15.64:0/2792696135 wait complete. 2024-05-08T22:03:42.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:42 smithi064 ceph-mon[42500]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 895 op/s 2024-05-08T22:03:42.547 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:42 smithi160 ceph-mon[46962]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 895 op/s 2024-05-08T22:03:42.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:42 smithi186 ceph-mon[46619]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 895 op/s 2024-05-08T22:03:44.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:44 smithi064 ceph-mon[42500]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 839 op/s 2024-05-08T22:03:44.548 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:44 smithi160 ceph-mon[46962]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 839 op/s 2024-05-08T22:03:44.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:44 smithi186 ceph-mon[46619]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 839 op/s 2024-05-08T22:03:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 535.0 s, 1898.6 tps, lat 52.618 ms stddev 60.718 2024-05-08T22:03:46.545 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:46 smithi160 ceph-mon[46962]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 17 MiB/s wr, 885 op/s 2024-05-08T22:03:46.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:46 smithi186 ceph-mon[46619]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 17 MiB/s wr, 885 op/s 2024-05-08T22:03:46.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:46 smithi064 ceph-mon[42500]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 17 MiB/s wr, 885 op/s 2024-05-08T22:03:48.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:48 smithi160 ceph-mon[46962]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 18 MiB/s wr, 859 op/s 2024-05-08T22:03:48.644 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:48 smithi186 ceph-mon[46619]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 18 MiB/s wr, 859 op/s 2024-05-08T22:03:48.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:48 smithi064 ceph-mon[42500]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 18 MiB/s wr, 859 op/s 2024-05-08T22:03:50.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:50 smithi160 ceph-mon[46962]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 15 MiB/s wr, 756 op/s 2024-05-08T22:03:50.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:50 smithi186 ceph-mon[46619]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 15 MiB/s wr, 756 op/s 2024-05-08T22:03:50.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:50 smithi064 ceph-mon[42500]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 15 MiB/s wr, 756 op/s 2024-05-08T22:03:51.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 540.0 s, 1563.4 tps, lat 63.257 ms stddev 74.361 2024-05-08T22:03:52.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:52 smithi160 ceph-mon[46962]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 771 op/s 2024-05-08T22:03:52.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:52 smithi186 ceph-mon[46619]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 771 op/s 2024-05-08T22:03:52.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:52 smithi064 ceph-mon[42500]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 771 op/s 2024-05-08T22:03:54.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:54 smithi186 ceph-mon[46619]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 714 op/s 2024-05-08T22:03:54.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:54 smithi064 ceph-mon[42500]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 714 op/s 2024-05-08T22:03:54.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:54 smithi160 ceph-mon[46962]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 714 op/s 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 -- 172.21.15.64:0/3843720039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 msgr2=0x7fa020108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 --2- 172.21.15.64:0/3843720039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa020108230 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa0180099c0 tx=0x7fa018009cd0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 -- 172.21.15.64:0/3843720039 shutdown_connections 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 --2- 172.21.15.64:0/3843720039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa020108230 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 --2- 172.21.15.64:0/3843720039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 0x7fa020100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 --2- 172.21.15.64:0/3843720039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa020100fe0 0x7fa020101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 -- 172.21.15.64:0/3843720039 >> 172.21.15.64:0/3843720039 conn(0x7fa0200fb360 msgr2=0x7fa0200fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 -- 172.21.15.64:0/3843720039 shutdown_connections 2024-05-08T22:03:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.008+0000 7fa02677f700 1 -- 172.21.15.64:0/3843720039 wait complete. 2024-05-08T22:03:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.009+0000 7fa02677f700 1 Processor -- start 2024-05-08T22:03:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 -- start start 2024-05-08T22:03:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 0x7fa02006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa020100fe0 0x7fa02006fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa0201b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020072d00 con 0x7fa020101980 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020072e70 con 0x7fa020100fe0 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.010+0000 7fa02677f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201b6330 con 0x7fa0200ffdf0 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa0201b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa0201b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46752/0 (socket says 172.21.15.64:46752) 2024-05-08T22:03:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa024d1c700 1 -- 172.21.15.64:0/920920058 learned_addr learned my addr 172.21.15.64:0/920920058 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa01ffff700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 0x7fa02006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa01ffff700 1 -- 172.21.15.64:0/920920058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa020100fe0 msgr2=0x7fa02006fc70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa01ffff700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa020100fe0 0x7fa02006fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa01ffff700 1 -- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 msgr2=0x7fa0201b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa01ffff700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa0201b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.011+0000 7fa01ffff700 1 -- 172.21.15.64:0/920920058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008009520 con 0x7fa0200ffdf0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.012+0000 7fa01ffff700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 0x7fa02006f730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa01000eee0 tx=0x7fa01000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.012+0000 7fa01dffb700 1 -- 172.21.15.64:0/920920058 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010018070 con 0x7fa0200ffdf0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.012+0000 7fa01dffb700 1 -- 172.21.15.64:0/920920058 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01000f040 con 0x7fa0200ffdf0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.012+0000 7fa01dffb700 1 -- 172.21.15.64:0/920920058 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010004310 con 0x7fa0200ffdf0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.012+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0180095f0 con 0x7fa0200ffdf0 2024-05-08T22:03:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.012+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201b6820 con 0x7fa0200ffdf0 2024-05-08T22:03:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.014+0000 7fa01dffb700 1 -- 172.21.15.64:0/920920058 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa01000cad0 con 0x7fa0200ffdf0 2024-05-08T22:03:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.015+0000 7fa01dffb700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa00c0a1180 0x7fa00c0a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.015+0000 7fa0177fe700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa00c0a1180 0x7fa00c0a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.015+0000 7fa01dffb700 1 -- 172.21.15.64:0/920920058 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0100cc950 con 0x7fa0200ffdf0 2024-05-08T22:03:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.016+0000 7fa032939b80 1 Processor -- start 2024-05-08T22:03:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.016+0000 7fa0177fe700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa00c0a1180 0x7fa00c0a3640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa008009ca0 tx=0x7fa008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.016+0000 7fa032939b80 1 -- start start 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa032939b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa032939b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa032939b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 0x55d26d9b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa032939b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d90e0f0 con 0x55d26d9b73e0 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa032939b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d90bab0 con 0x55d26d9b82d0 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa032939b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9073a0 con 0x55d26d9b6a80 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 0x55d26d9b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 0x55d26d9b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47620/0 (socket says 172.21.15.64:47620) 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 -- 172.21.15.64:0/2948102515 learned_addr learned my addr 172.21.15.64:0/2948102515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa0177fe700 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 -- 172.21.15.64:0/2948102515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 msgr2=0x55d26d9b6ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9b6ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 -- 172.21.15.64:0/2948102515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 msgr2=0x55d26d9b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.017+0000 7fa024d1c700 1 -- 172.21.15.64:0/2948102515 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26d788350 con 0x55d26d9b82d0 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.018+0000 7fa024d1c700 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 0x55d26d9b8750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa018006010 tx=0x7fa018009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f7d2a0ca6dbb53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.018+0000 7fa0167fc700 1 -- 172.21.15.64:0/2948102515 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018005270 con 0x55d26d9b82d0 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.018+0000 7fa0167fc700 1 -- 172.21.15.64:0/2948102515 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01803f040 con 0x55d26d9b82d0 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.018+0000 7fa0167fc700 1 -- 172.21.15.64:0/2948102515 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018003bf0 con 0x55d26d9b82d0 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.019+0000 7fa032939b80 1 -- 172.21.15.64:0/2948102515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 msgr2=0x55d26d9b8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.019+0000 7fa032939b80 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 0x55d26d9b8750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa018006010 tx=0x7fa018009530 comp rx=0 tx=0).stop 2024-05-08T22:03:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.020+0000 7fa032939b80 1 -- 172.21.15.64:0/2948102515 shutdown_connections 2024-05-08T22:03:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.020+0000 7fa032939b80 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b82d0 0x55d26d9b8750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.020+0000 7fa032939b80 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.020+0000 7fa032939b80 1 --2- 172.21.15.64:0/2948102515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.020+0000 7fa032939b80 1 -- 172.21.15.64:0/2948102515 >> 172.21.15.64:0/2948102515 conn(0x55d26d9151b0 msgr2=0x55d26d9b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.021+0000 7fa032939b80 1 -- 172.21.15.64:0/2948102515 shutdown_connections 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.021+0000 7fa032939b80 1 -- 172.21.15.64:0/2948102515 wait complete. 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.021+0000 7fa032939b80 1 -- start start 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.021+0000 7fa032939b80 1 Processor -- start 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.021+0000 7fa032939b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa032939b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b73e0 0x55d26d9c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa032939b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa032939b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d8a3500 con 0x55d26d9b82d0 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa032939b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d8a31a0 con 0x55d26d9b6a80 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa032939b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d78a640 con 0x55d26d9b73e0 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa032939b80 10 client.0 _ll_register_callbacks cb 0x55d26d8acad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46760/0 (socket says 172.21.15.64:46760) 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa024d1c700 1 -- 172.21.15.64:0/331806747 learned_addr learned my addr 172.21.15.64:0/331806747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa01ffff700 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa01ffff700 1 -- 172.21.15.64:0/331806747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b73e0 msgr2=0x55d26d9c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa01ffff700 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b73e0 0x55d26d9c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa01ffff700 1 -- 172.21.15.64:0/331806747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 msgr2=0x55d26d9c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa01ffff700 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.022+0000 7fa01ffff700 1 -- 172.21.15.64:0/331806747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa020108b80 con 0x55d26d9b6a80 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7fa01ffff700 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c0890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0100ca400 tx=0x7fa01009a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ecaef389f17397 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7f9fff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7f9fff7fe700 1 -- 172.21.15.64:0/331806747 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010099de0 con 0x55d26d9b6a80 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7f9fff7fe700 1 -- 172.21.15.64:0/331806747 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa01009dc10 con 0x55d26d9b6a80 2024-05-08T22:03:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d26d860210 con 0x55d26d9b6a80 2024-05-08T22:03:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7f9fff7fe700 1 -- 172.21.15.64:0/331806747 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0100a21d0 con 0x55d26d9b6a80 2024-05-08T22:03:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7fa032939b80 10 client.25667 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.023+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d26d734b00 con 0x55d26d9b6a80 2024-05-08T22:03:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7f9fff7fe700 1 -- 172.21.15.64:0/331806747 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0100b8030 con 0x55d26d9b6a80 2024-05-08T22:03:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7f9fff7fe700 1 -- 172.21.15.64:0/331806747 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa0100a1d40 con 0x55d26d9b6a80 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa032939b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa032939b80 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa032939b80 20 client.25667 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa032939b80 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26d9cc740 0x55d26d9ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa032939b80 4 client.25667 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d26d9867b0 con 0x55d26d9cc740 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7fa0177fe700 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26d9cc740 0x55d26d9ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.024+0000 7f9fff7fe700 10 client.25667 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.025+0000 7fa0177fe700 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26d9cc740 0x55d26d9ceb60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8c76d60b6b9479 server_cookie=1a9c5735b1d2348f in_seq=0 out_seq=0 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.025+0000 7f9fff7fe700 1 -- 172.21.15.64:0/331806747 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d26d9867b0 con 0x55d26d9cc740 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.025+0000 7f9fff7fe700 10 client.25667 handle_command_reply: tid=0 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.025+0000 7fa032939b80 1 client.25667 shutdown 2024-05-08T22:03:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 msgr2=0x55d26d9c0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c0890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0100ca400 tx=0x7fa01009a320 comp rx=0 tx=0).stop 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 shutdown_connections 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c0890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26d9cc740 0x55d26d9ceb60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 --2- 172.21.15.64:0/331806747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b73e0 0x55d26d9c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7f9fff7fe700 0 client.25667 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 >> 172.21.15.64:0/331806747 conn(0x55d26d9151b0 msgr2=0x55d26d9b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 shutdown_connections 2024-05-08T22:03:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 1 -- 172.21.15.64:0/331806747 wait complete. 2024-05-08T22:03:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.026+0000 7fa032939b80 20 client.25667 trim_cache size 0 max 16384 2024-05-08T22:03:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 Processor -- start 2024-05-08T22:03:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 -- start start 2024-05-08T22:03:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b73e0 0x55d26d9c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9ac7c0 con 0x55d26d9b82d0 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9c8f90 con 0x55d26d9b73e0 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.046+0000 7fa032939b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9c9b20 con 0x55d26d9b6a80 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:46250/0 (socket says 172.21.15.64:46250) 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 -- 172.21.15.64:0/1994707907 learned_addr learned my addr 172.21.15.64:0/1994707907 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa024d1c700 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 -- 172.21.15.64:0/1994707907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b73e0 msgr2=0x55d26d9c7dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b73e0 0x55d26d9c7dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 -- 172.21.15.64:0/1994707907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 msgr2=0x55d26d9b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 -- 172.21.15.64:0/1994707907 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26d8cd500 con 0x55d26d9b6a80 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.047+0000 7fa01ffff700 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9c84c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0100996d0 tx=0x7fa0100b81b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b08066e4ba8804 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa016ffd700 1 -- 172.21.15.64:0/1994707907 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100092e0 con 0x55d26d9b6a80 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa016ffd700 1 -- 172.21.15.64:0/1994707907 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0100b8040 con 0x55d26d9b6a80 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa016ffd700 1 -- 172.21.15.64:0/1994707907 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cd80 con 0x55d26d9b6a80 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 -- 172.21.15.64:0/1994707907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 msgr2=0x55d26d9c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9c84c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0100996d0 tx=0x7fa0100b81b0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 -- 172.21.15.64:0/1994707907 shutdown_connections 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b82d0 0x55d26d9b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b6a80 0x55d26d9c84c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 --2- 172.21.15.64:0/1994707907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b73e0 0x55d26d9c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 -- 172.21.15.64:0/1994707907 >> 172.21.15.64:0/1994707907 conn(0x55d26d9151b0 msgr2=0x55d26d9be860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 -- 172.21.15.64:0/1994707907 shutdown_connections 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.048+0000 7fa032939b80 1 -- 172.21.15.64:0/1994707907 wait complete. 2024-05-08T22:03:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 -- start start 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 Processor -- start 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 0x55d26d9c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9c7380 con 0x55d26d9b73e0 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9c74f0 con 0x55d26d9b6a80 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d9c76b0 con 0x55d26d9b82d0 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa032939b80 10 client.0 _ll_register_callbacks cb 0x55d26d8acad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.049+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 0x55d26d9c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa024d1c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 0x55d26d9c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:46258/0 (socket says 172.21.15.64:46258) 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa024d1c700 1 -- 172.21.15.64:0/3096793468 learned_addr learned my addr 172.21.15.64:0/3096793468 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47632/0 (socket says 172.21.15.64:47632) 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa0177fe700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 -- 172.21.15.64:0/3096793468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 msgr2=0x55d26d9c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 0x55d26d9c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 -- 172.21.15.64:0/3096793468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 msgr2=0x55d26d9c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 -- 172.21.15.64:0/3096793468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d26d734b00 con 0x55d26d9b6a80 2024-05-08T22:03:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa024d1c700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 0x55d26d9c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:03:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa01ffff700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c5940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55d26d9c19e0 tx=0x7fa010000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b8c5016c3c8104 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7f9fff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:03:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7f9fff7fe700 1 -- 172.21.15.64:0/3096793468 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01009dc10 con 0x55d26d9b6a80 2024-05-08T22:03:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa01803f040 con 0x55d26d9b6a80 2024-05-08T22:03:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.050+0000 7f9fff7fe700 1 -- 172.21.15.64:0/3096793468 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa01000cd80 con 0x55d26d9b6a80 2024-05-08T22:03:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.052+0000 7f9fff7fe700 1 -- 172.21.15.64:0/3096793468 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa010095bd0 con 0x55d26d9b6a80 2024-05-08T22:03:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 10 client.25670 fetch_fsmap learned FSMap version 41 2024-05-08T22:03:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d26d78cc70 con 0x55d26d9b6a80 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7f9fff7fe700 1 -- 172.21.15.64:0/3096793468 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01803f040 con 0x55d26d9b6a80 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7f9fff7fe700 1 -- 172.21.15.64:0/3096793468 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa010098e60 con 0x55d26d9b6a80 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 20 client.25670 populate_metadata read hostname 'smithi064' 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26da23380 0x55d26da25770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 4 client.25670 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.054+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d26d8ffe60 con 0x55d26da23380 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.055+0000 7fa0177fe700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26da23380 0x55d26da25770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.055+0000 7fa0177fe700 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26da23380 0x55d26da25770 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200cdf6fda1321b server_cookie=352f7c0dac35bfe7 in_seq=0 out_seq=0 2024-05-08T22:03:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.055+0000 7f9fff7fe700 10 client.25670 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.096+0000 7f9fff7fe700 1 -- 172.21.15.64:0/3096793468 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d26d8ffe60 con 0x55d26da23380 2024-05-08T22:03:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.096+0000 7f9fff7fe700 10 client.25670 handle_command_reply: tid=0 2024-05-08T22:03:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.097+0000 7fa032939b80 1 client.25670 shutdown 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 msgr2=0x55d26d9c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c5940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55d26d9c19e0 tx=0x7fa010000f40 comp rx=0 tx=0).stop 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 shutdown_connections 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d26d9b6a80 0x55d26d9c5940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55d26da23380 0x55d26da25770 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d26d9b82d0 0x55d26d9c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 --2- 172.21.15.64:0/3096793468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d26d9b73e0 0x55d26d9c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 >> 172.21.15.64:0/3096793468 conn(0x55d26d9151b0 msgr2=0x55d26d9bfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.104+0000 7f9fff7fe700 0 client.25670 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:03:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.105+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 shutdown_connections 2024-05-08T22:03:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.105+0000 7fa032939b80 1 -- 172.21.15.64:0/3096793468 wait complete. 2024-05-08T22:03:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa032939b80 20 client.25670 trim_cache size 0 max 16384 2024-05-08T22:03:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:03:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:03:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:03:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:03:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa00c0a1180 msgr2=0x7fa00c0a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa00c0a1180 0x7fa00c0a3640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa008009ca0 tx=0x7fa008009260 comp rx=0 tx=0).stop 2024-05-08T22:03:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 msgr2=0x7fa02006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 0x7fa02006f730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa01000eee0 tx=0x7fa01000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 shutdown_connections 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa00c0a1180 0x7fa00c0a3640 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa020101980 0x7fa0201b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0200ffdf0 0x7fa02006f730 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 --2- 172.21.15.64:0/920920058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa020100fe0 0x7fa02006fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 >> 172.21.15.64:0/920920058 conn(0x7fa0200fb360 msgr2=0x7fa020104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 shutdown_connections 2024-05-08T22:03:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:03:55.107+0000 7fa02677f700 1 -- 172.21.15.64:0/920920058 wait complete. 2024-05-08T22:03:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 545.0 s, 1392.0 tps, lat 72.760 ms stddev 117.951 2024-05-08T22:03:56.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:56 smithi186 ceph-mon[46619]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.04k op/s 2024-05-08T22:03:56.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:56 smithi064 ceph-mon[42500]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.04k op/s 2024-05-08T22:03:56.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:56 smithi160 ceph-mon[46962]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.04k op/s 2024-05-08T22:03:58.641 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:03:58 smithi186 ceph-mon[46619]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.45k op/s 2024-05-08T22:03:58.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:03:58 smithi064 ceph-mon[42500]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.45k op/s 2024-05-08T22:03:58.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:03:58 smithi160 ceph-mon[46962]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.45k op/s 2024-05-08T22:04:00.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:00 smithi064 ceph-mon[42500]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.76k op/s 2024-05-08T22:04:00.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:00 smithi160 ceph-mon[46962]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.76k op/s 2024-05-08T22:04:00.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:00 smithi186 ceph-mon[46619]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.76k op/s 2024-05-08T22:04:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 550.0 s, 1493.8 tps, lat 66.792 ms stddev 75.719 2024-05-08T22:04:02.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:02 smithi064 ceph-mon[42500]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 2.09k op/s 2024-05-08T22:04:02.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:02 smithi160 ceph-mon[46962]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 2.09k op/s 2024-05-08T22:04:02.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:02 smithi186 ceph-mon[46619]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 2.09k op/s 2024-05-08T22:04:04.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:04 smithi064 ceph-mon[42500]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 2.06k op/s 2024-05-08T22:04:04.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:04 smithi160 ceph-mon[46962]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 2.06k op/s 2024-05-08T22:04:04.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:04 smithi186 ceph-mon[46619]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 2.06k op/s 2024-05-08T22:04:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 555.0 s, 1983.0 tps, lat 50.430 ms stddev 57.977 2024-05-08T22:04:06.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:06 smithi064 ceph-mon[42500]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2024-05-08T22:04:06.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:06 smithi160 ceph-mon[46962]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2024-05-08T22:04:06.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:06 smithi186 ceph-mon[46619]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2024-05-08T22:04:07.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:07 smithi160 ceph-mon[46962]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2024-05-08T22:04:07.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:07 smithi186 ceph-mon[46619]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2024-05-08T22:04:07.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:07 smithi064 ceph-mon[42500]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2024-05-08T22:04:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.293+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1958962504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 msgr2=0x7f3fcc0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.293+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1958962504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc0731c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0007570 tx=0x7f3fc0007880 comp rx=0 tx=0).stop 2024-05-08T22:04:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1958962504 shutdown_connections 2024-05-08T22:04:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1958962504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc0731c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1958962504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 0x7f3fcc07ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1958962504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fcc075cd0 0x7f3fcc078160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1958962504 >> 172.21.15.64:0/1958962504 conn(0x7f3fcc06df30 msgr2=0x7f3fcc070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1958962504 shutdown_connections 2024-05-08T22:04:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1958962504 wait complete. 2024-05-08T22:04:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.294+0000 7f3fd2f31700 1 Processor -- start 2024-05-08T22:04:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.295+0000 7f3fd2f31700 1 -- start start 2024-05-08T22:04:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.296+0000 7f3fd2f31700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc1b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.296+0000 7f3fd2f31700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fcc075cd0 0x7f3fcc1b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.296+0000 7f3fd2f31700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 0x7f3fcc1b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.296+0000 7f3fd2f31700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1b40d0 con 0x7f3fcc0786a0 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.296+0000 7f3fd2f31700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1b4240 con 0x7f3fcc072da0 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.296+0000 7f3fd2f31700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1b4400 con 0x7f3fcc075cd0 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.297+0000 7f3fd14ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 0x7f3fcc1b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.297+0000 7f3fd14ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 0x7f3fcc1b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54252/0 (socket says 172.21.15.64:54252) 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.297+0000 7f3fd14ce700 1 -- 172.21.15.64:0/1484449209 learned_addr learned my addr 172.21.15.64:0/1484449209 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.297+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc1b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.297+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fcc075cd0 0x7f3fcc1b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.298+0000 7f3fd0ccd700 1 -- 172.21.15.64:0/1484449209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fcc075cd0 msgr2=0x7f3fcc1b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.298+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fcc075cd0 0x7f3fcc1b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.298+0000 7f3fd0ccd700 1 -- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 msgr2=0x7f3fcc1b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.298+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 0x7f3fcc1b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.298+0000 7f3fd0ccd700 1 -- 172.21.15.64:0/1484449209 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0007220 con 0x7f3fcc072da0 2024-05-08T22:04:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.299+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc1b0ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000a140 tx=0x7f3fc000a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.299+0000 7f3fc9ffb700 1 -- 172.21.15.64:0/1484449209 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000f070 con 0x7f3fcc072da0 2024-05-08T22:04:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.299+0000 7f3fc9ffb700 1 -- 172.21.15.64:0/1484449209 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc0007e60 con 0x7f3fcc072da0 2024-05-08T22:04:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.299+0000 7f3fc9ffb700 1 -- 172.21.15.64:0/1484449209 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000c860 con 0x7f3fcc072da0 2024-05-08T22:04:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.299+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc1b4680 con 0x7f3fcc072da0 2024-05-08T22:04:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.300+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1b4ae0 con 0x7f3fcc072da0 2024-05-08T22:04:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 Processor -- start 2024-05-08T22:04:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 -- start start 2024-05-08T22:04:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9230f0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d920ab0 con 0x55ba4d9dca90 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.305+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d91c3a0 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd0ccd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9dceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:59176/0 (socket says 172.21.15.64:59176) 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 -- 172.21.15.64:0/1557183851 learned_addr learned my addr 172.21.15.64:0/1557183851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 -- 172.21.15.64:0/1557183851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 msgr2=0x55ba4d9ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 -- 172.21.15.64:0/1557183851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 msgr2=0x55ba4d9de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 -- 172.21.15.64:0/1557183851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4d79d350 con 0x55ba4d9dca90 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9dceb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009f60 tx=0x7f3fc4009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfa2d29c570ad3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.306+0000 7f3fb6ffd700 1 -- 172.21.15.64:0/1557183851 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4007e50 con 0x55ba4d9dca90 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.307+0000 7f3fb6ffd700 1 -- 172.21.15.64:0/1557183851 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc4036ac0 con 0x55ba4d9dca90 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.307+0000 7f3fb6ffd700 1 -- 172.21.15.64:0/1557183851 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40359f0 con 0x55ba4d9dca90 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.307+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/1557183851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 msgr2=0x55ba4d9dceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.307+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9dceb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009f60 tx=0x7f3fc4009230 comp rx=0 tx=0).stop 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.309+0000 7f3fc9ffb700 1 -- 172.21.15.64:0/1484449209 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3fc00048f0 con 0x7f3fcc072da0 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/1557183851 shutdown_connections 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9dceb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/1557183851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/1557183851 >> 172.21.15.64:0/1557183851 conn(0x55ba4d92a1b0 msgr2=0x55ba4d9c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/1557183851 shutdown_connections 2024-05-08T22:04:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/1557183851 wait complete. 2024-05-08T22:04:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.310+0000 7f3fdf0ebb80 1 -- start start 2024-05-08T22:04:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.311+0000 7f3fdf0ebb80 1 Processor -- start 2024-05-08T22:04:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.311+0000 7f3fc9ffb700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3fbc0a1130 0x7f3fbc0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.311+0000 7f3fc9ffb700 1 -- 172.21.15.64:0/1484449209 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3fc00cd0d0 con 0x7f3fcc072da0 2024-05-08T22:04:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d8b8500 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d8b81a0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d79f640 con 0x55ba4d9dca90 2024-05-08T22:04:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fdf0ebb80 10 client.0 _ll_register_callbacks cb 0x55ba4d8c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3fbc0a1130 0x7f3fbc0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54270/0 (socket says 172.21.15.64:54270) 2024-05-08T22:04:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fcbfff700 1 -- 172.21.15.64:0/300860474 learned_addr learned my addr 172.21.15.64:0/300860474 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.312+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fcbfff700 1 -- 172.21.15.64:0/300860474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 msgr2=0x55ba4d9cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fcbfff700 1 -- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 msgr2=0x55ba4d9d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fcbfff700 1 -- 172.21.15.64:0/300860474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc4009b90 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:04:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:04:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9cd3c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8012840 tx=0x7f3fb8012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4596a8d735f42120 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fabfff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:04:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fabfff700 1 -- 172.21.15.64:0/300860474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8010880 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.313+0000 7f3fabfff700 1 -- 172.21.15.64:0/300860474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3fb80109e0 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.314+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba4d875210 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.314+0000 7f3fabfff700 1 -- 172.21.15.64:0/300860474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb8010ba0 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.314+0000 7f3fdf0ebb80 10 client.15912 fetch_fsmap learned FSMap version 41 2024-05-08T22:04:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.314+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba4d749b00 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.314+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3fbc0a1130 0x7f3fbc0a35f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4035490 tx=0x7f3fc40090b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fabfff700 1 -- 172.21.15.64:0/300860474 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba4d875210 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fabfff700 1 -- 172.21.15.64:0/300860474 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3fb80193b0 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fdf0ebb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:04:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fdf0ebb80 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fdf0ebb80 20 client.15912 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4d9d8d60 0x55ba4d9db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fdf0ebb80 4 client.15912 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:04:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba4d99b7b0 con 0x55ba4d9d8d60 2024-05-08T22:04:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.315+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4d9d8d60 0x55ba4d9db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.316+0000 7f3fabfff700 10 client.15912 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.316+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4d9d8d60 0x55ba4d9db180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3101aae484ef091e server_cookie=cb009aecfd77703a in_seq=0 out_seq=0 2024-05-08T22:04:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.317+0000 7f3fabfff700 1 -- 172.21.15.64:0/300860474 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba4d99b7b0 con 0x55ba4d9d8d60 2024-05-08T22:04:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.317+0000 7f3fabfff700 10 client.15912 handle_command_reply: tid=0 2024-05-08T22:04:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.317+0000 7f3fdf0ebb80 1 client.15912 shutdown 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.317+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 msgr2=0x55ba4d9cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.317+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9cd3c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8012840 tx=0x7f3fb8012c00 comp rx=0 tx=0).stop 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.320+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 shutdown_connections 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.320+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.320+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4d9d8d60 0x55ba4d9db180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.320+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.321+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/300860474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9cd3c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.321+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 >> 172.21.15.64:0/300860474 conn(0x55ba4d92a1b0 msgr2=0x55ba4d9c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.321+0000 7f3fabfff700 0 client.15912 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.322+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 shutdown_connections 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.322+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/300860474 wait complete. 2024-05-08T22:04:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.322+0000 7f3fdf0ebb80 20 client.15912 trim_cache size 0 max 16384 2024-05-08T22:04:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 Processor -- start 2024-05-08T22:04:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 -- start start 2024-05-08T22:04:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9c1330 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9cb5e0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9cc9f0 con 0x55ba4d9dca90 2024-05-08T22:04:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54280/0 (socket says 172.21.15.64:54280) 2024-05-08T22:04:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 -- 172.21.15.64:0/3370800349 learned_addr learned my addr 172.21.15.64:0/3370800349 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 -- 172.21.15.64:0/3370800349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 msgr2=0x55ba4d9db150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9db150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 -- 172.21.15.64:0/3370800349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 msgr2=0x55ba4d9d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fcbfff700 1 -- 172.21.15.64:0/3370800349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4d8e2500 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.342+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.343+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4ab0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8023020 tx=0x7f3fb8005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cf6273b5564b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.345+0000 7f3fb77fe700 1 -- 172.21.15.64:0/3370800349 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80193b0 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.345+0000 7f3fb77fe700 1 -- 172.21.15.64:0/3370800349 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb8019510 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.345+0000 7f3fb77fe700 1 -- 172.21.15.64:0/3370800349 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80235b0 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3370800349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 msgr2=0x55ba4d9d4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4ab0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8023020 tx=0x7f3fb8005930 comp rx=0 tx=0).stop 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3370800349 shutdown_connections 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dca90 0x55ba4d9db150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3370800349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4ab0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3370800349 >> 172.21.15.64:0/3370800349 conn(0x55ba4d92a1b0 msgr2=0x55ba4d9e4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3370800349 shutdown_connections 2024-05-08T22:04:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3370800349 wait complete. 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.346+0000 7f3fdf0ebb80 1 -- start start 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 Processor -- start 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9d0f00 con 0x55ba4d9de2e0 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9d1070 con 0x55ba4d9dca90 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4d9d1230 con 0x55ba4d9dd3f0 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.347+0000 7f3fdf0ebb80 10 client.0 _ll_register_callbacks cb 0x55ba4d8c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd0ccd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48594/0 (socket says 172.21.15.64:48594) 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fcbfff700 1 -- 172.21.15.64:0/3734160876 learned_addr learned my addr 172.21.15.64:0/3734160876 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd14ce700 1 -- 172.21.15.64:0/3734160876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 msgr2=0x55ba4d9d4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd14ce700 1 -- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 msgr2=0x55ba4d9d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd14ce700 1 -- 172.21.15.64:0/3734160876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba4d79d350 con 0x55ba4d9de2e0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.348+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fd14ce700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d0920 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009550 tx=0x7f3fc4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212661a7b9119638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fabfff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fabfff700 1 -- 172.21.15.64:0/3734160876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4035ad0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fabfff700 1 -- 172.21.15.64:0/3734160876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3fc40354c0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fc4036ac0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fabfff700 1 -- 172.21.15.64:0/3734160876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc4035c30 con 0x55ba4d9de2e0 2024-05-08T22:04:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fdf0ebb80 10 client.15918 fetch_fsmap learned FSMap version 41 2024-05-08T22:04:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.349+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba4d749b00 con 0x55ba4d9de2e0 2024-05-08T22:04:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fabfff700 1 -- 172.21.15.64:0/3734160876 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc4036ac0 con 0x55ba4d9de2e0 2024-05-08T22:04:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fabfff700 1 -- 172.21.15.64:0/3734160876 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3fc405d070 con 0x55ba4d9de2e0 2024-05-08T22:04:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fdf0ebb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:04:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fdf0ebb80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fdf0ebb80 20 client.15918 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4da37f30 0x55ba4da3a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fdf0ebb80 4 client.15918 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba4d914e60 con 0x55ba4da37f30 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4da37f30 0x55ba4da3a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fabfff700 10 client.15918 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.350+0000 7f3fd0ccd700 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4da37f30 0x55ba4da3a390 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900dc2b32e9e400c server_cookie=618ad876bee8e20d in_seq=0 out_seq=0 2024-05-08T22:04:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.358+0000 7f3fabfff700 1 -- 172.21.15.64:0/3734160876 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba4d914e60 con 0x55ba4da37f30 2024-05-08T22:04:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.358+0000 7f3fabfff700 10 client.15918 handle_command_reply: tid=0 2024-05-08T22:04:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.358+0000 7f3fdf0ebb80 1 client.15918 shutdown 2024-05-08T22:04:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.358+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 msgr2=0x55ba4d9d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.358+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d0920 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009550 tx=0x7f3fc4009f90 comp rx=0 tx=0).stop 2024-05-08T22:04:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 shutdown_connections 2024-05-08T22:04:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba4d9dca90 0x55ba4d9d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ba4da37f30 0x55ba4da3a390 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba4d9de2e0 0x55ba4d9d0920 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fabfff700 0 client.15918 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 --2- 172.21.15.64:0/3734160876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba4d9dd3f0 0x55ba4d9d4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 >> 172.21.15.64:0/3734160876 conn(0x55ba4d92a1b0 msgr2=0x55ba4d9e9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 shutdown_connections 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 1 -- 172.21.15.64:0/3734160876 wait complete. 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.359+0000 7f3fdf0ebb80 20 client.15918 trim_cache size 0 max 16384 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:04:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:04:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3fbc0a1130 msgr2=0x7f3fbc0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3fbc0a1130 0x7f3fbc0a35f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4035490 tx=0x7f3fc40090b0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 msgr2=0x7f3fcc1b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc1b0ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000a140 tx=0x7f3fc000a5c0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 shutdown_connections 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f3fbc0a1130 0x7f3fbc0a35f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fcc072da0 0x7f3fcc1b0ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0786a0 0x7f3fcc1b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 --2- 172.21.15.64:0/1484449209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fcc075cd0 0x7f3fcc1b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 >> 172.21.15.64:0/1484449209 conn(0x7f3fcc06df30 msgr2=0x7f3fcc077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 shutdown_connections 2024-05-08T22:04:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:08.360+0000 7f3fd2f31700 1 -- 172.21.15.64:0/1484449209 wait complete. 2024-05-08T22:04:09.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:09 smithi160 ceph-mon[46962]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 1.27k op/s 2024-05-08T22:04:09.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:09 smithi186 ceph-mon[46619]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 1.27k op/s 2024-05-08T22:04:09.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:09 smithi064 ceph-mon[42500]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 1.27k op/s 2024-05-08T22:04:11.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 560.0 s, 1777.6 tps, lat 55.978 ms stddev 65.170 2024-05-08T22:04:11.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:11 smithi160 ceph-mon[46962]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-05-08T22:04:11.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:11 smithi186 ceph-mon[46619]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-05-08T22:04:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:11 smithi064 ceph-mon[42500]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-05-08T22:04:12.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:12 smithi160 ceph-mon[46962]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 720 op/s 2024-05-08T22:04:12.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:12 smithi186 ceph-mon[46619]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 720 op/s 2024-05-08T22:04:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:12 smithi064 ceph-mon[42500]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 720 op/s 2024-05-08T22:04:15.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:15 smithi160 ceph-mon[46962]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 707 op/s 2024-05-08T22:04:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:15 smithi186 ceph-mon[46619]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 707 op/s 2024-05-08T22:04:15.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:15 smithi064 ceph-mon[42500]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 707 op/s 2024-05-08T22:04:16.486 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 565.0 s, 1607.9 tps, lat 62.530 ms stddev 67.630 2024-05-08T22:04:16.898 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:16 smithi186 ceph-mon[46619]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 710 op/s 2024-05-08T22:04:16.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:16 smithi064 ceph-mon[42500]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 710 op/s 2024-05-08T22:04:17.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:16 smithi160 ceph-mon[46962]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 710 op/s 2024-05-08T22:04:18.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:18 smithi160 ceph-mon[46962]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 654 op/s 2024-05-08T22:04:18.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:18 smithi186 ceph-mon[46619]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 654 op/s 2024-05-08T22:04:18.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:18 smithi064 ceph-mon[42500]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 654 op/s 2024-05-08T22:04:20.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:20 smithi160 ceph-mon[46962]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-05-08T22:04:20.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:20 smithi186 ceph-mon[46619]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-05-08T22:04:20.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:20 smithi064 ceph-mon[42500]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-05-08T22:04:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 570.0 s, 2041.2 tps, lat 49.257 ms stddev 56.127 2024-05-08T22:04:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 -- 172.21.15.64:0/2628001342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d8101990 msgr2=0x7f15d8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 --2- 172.21.15.64:0/2628001342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d8101990 0x7f15d8108240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b0d0 tx=0x7f15d000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 -- 172.21.15.64:0/2628001342 shutdown_connections 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 --2- 172.21.15.64:0/2628001342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d8101990 0x7f15d8108240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 --2- 172.21.15.64:0/2628001342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15d80ffe00 0x7f15d8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 --2- 172.21.15.64:0/2628001342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d8100ff0 0x7f15d8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.547+0000 7f15dee98700 1 -- 172.21.15.64:0/2628001342 >> 172.21.15.64:0/2628001342 conn(0x7f15d80fb390 msgr2=0x7f15d80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.548+0000 7f15dee98700 1 -- 172.21.15.64:0/2628001342 shutdown_connections 2024-05-08T22:04:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.548+0000 7f15dee98700 1 -- 172.21.15.64:0/2628001342 wait complete. 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.548+0000 7f15dee98700 1 Processor -- start 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.548+0000 7f15dee98700 1 -- start start 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dee98700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 0x7f15d819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dee98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d8100ff0 0x7f15d819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dee98700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15d8101990 0x7f15d81a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dee98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81a4fc0 con 0x7f15d8100ff0 2024-05-08T22:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dee98700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81a5130 con 0x7f15d80ffe00 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dee98700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81a52f0 con 0x7f15d8101990 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 0x7f15d819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 0x7f15d819d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54402/0 (socket says 172.21.15.64:54402) 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 -- 172.21.15.64:0/3138874739 learned_addr learned my addr 172.21.15.64:0/3138874739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 -- 172.21.15.64:0/3138874739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15d8101990 msgr2=0x7f15d81a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15d8101990 0x7f15d81a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 -- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d8100ff0 msgr2=0x7f15d819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d8100ff0 0x7f15d819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.549+0000 7f15dcc34700 1 -- 172.21.15.64:0/3138874739 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009d00 con 0x7f15d80ffe00 2024-05-08T22:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.550+0000 7f15dcc34700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 0x7f15d819d4e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00d980 tx=0x7f15cc00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.555+0000 7f15d5ffb700 1 -- 172.21.15.64:0/3138874739 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00de60 con 0x7f15d80ffe00 2024-05-08T22:04:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.555+0000 7f15d5ffb700 1 -- 172.21.15.64:0/3138874739 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15cc0044e0 con 0x7f15d80ffe00 2024-05-08T22:04:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.555+0000 7f15d5ffb700 1 -- 172.21.15.64:0/3138874739 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc003be0 con 0x7f15d80ffe00 2024-05-08T22:04:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.555+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d81a55d0 con 0x7f15d80ffe00 2024-05-08T22:04:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.555+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d81a5a90 con 0x7f15d80ffe00 2024-05-08T22:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.561+0000 7f15eb052b80 1 Processor -- start 2024-05-08T22:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.561+0000 7f15eb052b80 1 -- start start 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x559682860eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682861d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828622e0 0x559682862760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15eb052b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596827a70f0 con 0x5596828622e0 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15eb052b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596827a4ab0 con 0x559682860a90 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15eb052b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596827a03a0 con 0x5596828613f0 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828622e0 0x559682862760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828622e0 0x559682862760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57008/0 (socket says 172.21.15.64:57008) 2024-05-08T22:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.562+0000 7f15d7fff700 1 -- 172.21.15.64:0/3132718149 learned_addr learned my addr 172.21.15.64:0/3132718149 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.563+0000 7f15dd435700 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x559682860eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.564+0000 7f15d5ffb700 1 -- 172.21.15.64:0/3138874739 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f15cc009940 con 0x7f15d80ffe00 2024-05-08T22:04:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.564+0000 7f15d5ffb700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15c80a12f0 0x7f15c80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.565+0000 7f15d5ffb700 1 -- 172.21.15.64:0/3138874739 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15cc0cb890 con 0x7f15d80ffe00 2024-05-08T22:04:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.565+0000 7f15dd435700 1 -- 172.21.15.64:0/3132718149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 msgr2=0x559682861d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.565+0000 7f15dd435700 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682861d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.565+0000 7f15dd435700 1 -- 172.21.15.64:0/3132718149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828622e0 msgr2=0x559682862760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.565+0000 7f15dd435700 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828622e0 0x559682862760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.565+0000 7f15dd435700 1 -- 172.21.15.64:0/3132718149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559682621350 con 0x559682860a90 2024-05-08T22:04:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.570+0000 7f15dd435700 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x559682860eb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15d0006070 tx=0x7f15d0005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a00dc675524aa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.573+0000 7f15c2ffd700 1 -- 172.21.15.64:0/3132718149 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00092d0 con 0x559682860a90 2024-05-08T22:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.573+0000 7f15c2ffd700 1 -- 172.21.15.64:0/3132718149 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d0009430 con 0x559682860a90 2024-05-08T22:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.573+0000 7f15c2ffd700 1 -- 172.21.15.64:0/3132718149 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00052d0 con 0x559682860a90 2024-05-08T22:04:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.574+0000 7f15dd435700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15c80a12f0 0x7f15c80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.576+0000 7f15eb052b80 1 -- 172.21.15.64:0/3132718149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 msgr2=0x559682860eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.576+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x559682860eb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15d0006070 tx=0x7f15d0005a40 comp rx=0 tx=0).stop 2024-05-08T22:04:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.576+0000 7f15dd435700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15c80a12f0 0x7f15c80a37b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005a10 tx=0x7f15d00057e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.581+0000 7f15eb052b80 1 -- 172.21.15.64:0/3132718149 shutdown_connections 2024-05-08T22:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.581+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x559682860eb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.581+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828622e0 0x559682862760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.581+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3132718149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682861d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.581+0000 7f15eb052b80 1 -- 172.21.15.64:0/3132718149 >> 172.21.15.64:0/3132718149 conn(0x5596827ae1b0 msgr2=0x55968284c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.583+0000 7f15eb052b80 1 -- 172.21.15.64:0/3132718149 shutdown_connections 2024-05-08T22:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.583+0000 7f15eb052b80 1 -- 172.21.15.64:0/3132718149 wait complete. 2024-05-08T22:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.583+0000 7f15eb052b80 1 -- start start 2024-05-08T22:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.583+0000 7f15eb052b80 1 Processor -- start 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x55968286a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 0x559682871e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968273c500 con 0x5596828613f0 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968273c1a0 con 0x559682860a90 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682623640 con 0x5596828622e0 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15eb052b80 10 client.0 _ll_register_callbacks cb 0x559682745ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15dd435700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 0x559682871e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15dd435700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 0x559682871e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:34126/0 (socket says 172.21.15.64:34126) 2024-05-08T22:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.584+0000 7f15dd435700 1 -- 172.21.15.64:0/2120196166 learned_addr learned my addr 172.21.15.64:0/2120196166 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dcc34700 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x55968286a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dd435700 1 -- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 msgr2=0x55968286a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dd435700 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x55968286a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dd435700 1 -- 172.21.15.64:0/2120196166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 msgr2=0x55968286ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dd435700 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dd435700 1 -- 172.21.15.64:0/2120196166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d8108b90 con 0x5596828622e0 2024-05-08T22:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15dd435700 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 0x559682871e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15d0007940 tx=0x7f15d00054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b026fa07dd392031 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15b7fff700 1 -- 172.21.15.64:0/2120196166 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000e030 con 0x5596828622e0 2024-05-08T22:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.585+0000 7f15b7fff700 1 -- 172.21.15.64:0/2120196166 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15d0007670 con 0x5596828622e0 2024-05-08T22:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.589+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596826f9210 con 0x5596828622e0 2024-05-08T22:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.589+0000 7f15b7fff700 1 -- 172.21.15.64:0/2120196166 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d0005260 con 0x5596828622e0 2024-05-08T22:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.592+0000 7f15eb052b80 10 client.25567 fetch_fsmap learned FSMap version 41 2024-05-08T22:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.592+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5596825cdb00 con 0x5596828622e0 2024-05-08T22:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.592+0000 7f15b7fff700 1 -- 172.21.15.64:0/2120196166 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596826f9210 con 0x5596828622e0 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15b7fff700 1 -- 172.21.15.64:0/2120196166 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f15d0019070 con 0x5596828622e0 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15eb052b80 10 client.25567 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15eb052b80 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15eb052b80 20 client.25567 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15eb052b80 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559682887500 0x559682889920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15eb052b80 4 client.25567 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55968281f7b0 con 0x559682887500 2024-05-08T22:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.593+0000 7f15dcc34700 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559682887500 0x559682889920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.594+0000 7f15b7fff700 10 client.25567 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.594+0000 7f15dcc34700 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559682887500 0x559682889920 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19982d6b453b7f7b server_cookie=a055e520cd168e8c in_seq=0 out_seq=0 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.594+0000 7f15b7fff700 1 -- 172.21.15.64:0/2120196166 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55968281f7b0 con 0x559682887500 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.594+0000 7f15b7fff700 10 client.25567 handle_command_reply: tid=0 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.594+0000 7f15eb052b80 1 client.25567 shutdown 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 msgr2=0x559682871e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 0x559682871e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15d0007940 tx=0x7f15d00054f0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 shutdown_connections 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559682860a90 0x55968286a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559682887500 0x559682889920 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828622e0 0x559682871e60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 --2- 172.21.15.64:0/2120196166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15b7fff700 0 client.25567 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 >> 172.21.15.64:0/2120196166 conn(0x5596827ae1b0 msgr2=0x55968284a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 shutdown_connections 2024-05-08T22:04:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 1 -- 172.21.15.64:0/2120196166 wait complete. 2024-05-08T22:04:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.595+0000 7f15eb052b80 20 client.25567 trim_cache size 0 max 16384 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 Processor -- start 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 -- start start 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559682860a90 0x559682883520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682871d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x5596828626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596828457c0 con 0x559682860a90 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968286b7e0 con 0x5596828622e0 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15eb052b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968286a450 con 0x5596828613f0 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.619+0000 7f15dd435700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x5596828626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x5596828626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54428/0 (socket says 172.21.15.64:54428) 2024-05-08T22:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 -- 172.21.15.64:0/3470104886 learned_addr learned my addr 172.21.15.64:0/3470104886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dcc34700 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559682860a90 0x559682883520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15d7fff700 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682871d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 -- 172.21.15.64:0/3470104886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 msgr2=0x559682871d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682871d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 -- 172.21.15.64:0/3470104886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559682860a90 msgr2=0x559682883520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559682860a90 0x559682883520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 -- 172.21.15.64:0/3470104886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559682766500 con 0x5596828622e0 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dcc34700 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559682860a90 0x559682883520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.620+0000 7f15dd435700 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x5596828626c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009400 tx=0x7f15d0007fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c7e75def3162d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.621+0000 7f15c37fe700 1 -- 172.21.15.64:0/3470104886 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0007970 con 0x5596828622e0 2024-05-08T22:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.621+0000 7f15c37fe700 1 -- 172.21.15.64:0/3470104886 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d000f040 con 0x5596828622e0 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.621+0000 7f15c37fe700 1 -- 172.21.15.64:0/3470104886 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0024610 con 0x5596828622e0 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 -- 172.21.15.64:0/3470104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 msgr2=0x5596828626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x5596828626c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009400 tx=0x7f15d0007fd0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 -- 172.21.15.64:0/3470104886 shutdown_connections 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x5596828626c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559682860a90 0x559682883520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 --2- 172.21.15.64:0/3470104886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596828613f0 0x559682871d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 -- 172.21.15.64:0/3470104886 >> 172.21.15.64:0/3470104886 conn(0x5596827ae1b0 msgr2=0x559682868850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 -- 172.21.15.64:0/3470104886 shutdown_connections 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 -- 172.21.15.64:0/3470104886 wait complete. 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.622+0000 7f15eb052b80 1 -- start start 2024-05-08T22:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 Processor -- start 2024-05-08T22:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559682860a90 0x55968286f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x559682870690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682870bd0 con 0x5596828613f0 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682870d40 con 0x5596828622e0 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559682870f00 con 0x559682860a90 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15eb052b80 10 client.0 _ll_register_callbacks cb 0x559682745ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15dd435700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x559682870690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.623+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57020/0 (socket says 172.21.15.64:57020) 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15d7fff700 1 -- 172.21.15.64:0/1987235000 learned_addr learned my addr 172.21.15.64:0/1987235000 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dcc34700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559682860a90 0x55968286f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dd435700 1 -- 172.21.15.64:0/1987235000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559682860a90 msgr2=0x55968286f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dd435700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559682860a90 0x55968286f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dd435700 1 -- 172.21.15.64:0/1987235000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 msgr2=0x55968286f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dd435700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dd435700 1 -- 172.21.15.64:0/1987235000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d81a5a90 con 0x5596828622e0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15d7fff700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15dd435700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x559682870690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009640 tx=0x7f15d0042ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3772beb43ec239da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15b7fff700 1 -- 172.21.15.64:0/1987235000 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0003c10 con 0x5596828622e0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15b7fff700 1 -- 172.21.15.64:0/1987235000 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15d0003e00 con 0x5596828622e0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.624+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15d0009430 con 0x5596828622e0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.625+0000 7f15b7fff700 1 -- 172.21.15.64:0/1987235000 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d00055e0 con 0x5596828622e0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.625+0000 7f15eb052b80 10 client.25700 fetch_fsmap learned FSMap version 41 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.625+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5596825cdb00 con 0x5596828622e0 2024-05-08T22:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.625+0000 7f15b7fff700 1 -- 172.21.15.64:0/1987235000 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d0009430 con 0x5596828622e0 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15b7fff700 1 -- 172.21.15.64:0/1987235000 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f15d0019040 con 0x5596828622e0 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15eb052b80 10 client.25700 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15eb052b80 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15eb052b80 20 client.25700 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15eb052b80 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5596828ddba0 0x5596828dff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15eb052b80 4 client.25700 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559682798e60 con 0x5596828ddba0 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.626+0000 7f15dcc34700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5596828ddba0 0x5596828dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.627+0000 7f15dcc34700 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5596828ddba0 0x5596828dff90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7feb10f0439fc27a server_cookie=5501c11af8540099 in_seq=0 out_seq=0 2024-05-08T22:04:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.627+0000 7f15b7fff700 10 client.25700 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15b7fff700 1 -- 172.21.15.64:0/1987235000 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559682798e60 con 0x5596828ddba0 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15b7fff700 10 client.25700 handle_command_reply: tid=0 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 client.25700 shutdown 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 msgr2=0x559682870690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x559682870690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009640 tx=0x7f15d0042ae0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 shutdown_connections 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596828622e0 0x559682870690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5596828ddba0 0x5596828dff90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559682860a90 0x55968286f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 --2- 172.21.15.64:0/1987235000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596828613f0 0x55968286f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 >> 172.21.15.64:0/1987235000 conn(0x5596827ae1b0 msgr2=0x559682865520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15b7fff700 0 client.25700 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.634+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 shutdown_connections 2024-05-08T22:04:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.635+0000 7f15eb052b80 1 -- 172.21.15.64:0/1987235000 wait complete. 2024-05-08T22:04:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.635+0000 7f15eb052b80 20 client.25700 trim_cache size 0 max 16384 2024-05-08T22:04:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:04:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:04:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:04:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:04:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.635+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15c80a12f0 msgr2=0x7f15c80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.635+0000 7f15dee98700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15c80a12f0 0x7f15c80a37b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005a10 tx=0x7f15d00057e0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 msgr2=0x7f15d819d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 0x7f15d819d4e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00d980 tx=0x7f15cc00dd40 comp rx=0 tx=0).stop 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 shutdown_connections 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f15c80a12f0 0x7f15c80a37b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15d80ffe00 0x7f15d819d4e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15d8101990 0x7f15d81a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 --2- 172.21.15.64:0/3138874739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d8100ff0 0x7f15d819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 >> 172.21.15.64:0/3138874739 conn(0x7f15d80fb390 msgr2=0x7f15d810d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 shutdown_connections 2024-05-08T22:04:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:21.636+0000 7f15dee98700 1 -- 172.21.15.64:0/3138874739 wait complete. 2024-05-08T22:04:22.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:22 smithi186 ceph-mon[46619]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 MiB/s wr, 634 op/s 2024-05-08T22:04:22.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:22 smithi064 ceph-mon[42500]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 MiB/s wr, 634 op/s 2024-05-08T22:04:23.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:22 smithi160 ceph-mon[46962]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 MiB/s wr, 634 op/s 2024-05-08T22:04:24.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:24 smithi160 ceph-mon[46962]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-08T22:04:24.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:24 smithi186 ceph-mon[46619]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-08T22:04:24.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:24 smithi064 ceph-mon[42500]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-08T22:04:26.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 575.0 s, 1863.8 tps, lat 53.519 ms stddev 58.858 2024-05-08T22:04:26.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:26 smithi160 ceph-mon[46962]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-08T22:04:26.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:26 smithi186 ceph-mon[46619]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-08T22:04:26.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:26 smithi064 ceph-mon[42500]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-08T22:04:28.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:28 smithi160 ceph-mon[46962]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 MiB/s wr, 613 op/s 2024-05-08T22:04:28.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:28 smithi186 ceph-mon[46619]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 MiB/s wr, 613 op/s 2024-05-08T22:04:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:28 smithi064 ceph-mon[42500]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 MiB/s wr, 613 op/s 2024-05-08T22:04:30.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:30 smithi160 ceph-mon[46962]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-05-08T22:04:30.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:30 smithi186 ceph-mon[46619]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-05-08T22:04:30.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:30 smithi064 ceph-mon[42500]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-05-08T22:04:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 580.0 s, 1439.4 tps, lat 69.440 ms stddev 93.994 2024-05-08T22:04:32.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:32 smithi160 ceph-mon[46962]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 686 op/s 2024-05-08T22:04:32.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:32 smithi186 ceph-mon[46619]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 686 op/s 2024-05-08T22:04:32.972 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:32 smithi064 ceph-mon[42500]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 686 op/s 2024-05-08T22:04:33.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:33 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:33 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:33.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:33 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:34.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:34 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:34.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:34 smithi160 ceph-mon[46962]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 759 op/s 2024-05-08T22:04:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.862+0000 7f258d2c4700 1 -- 172.21.15.64:0/3066134853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 msgr2=0x7f258810d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.862+0000 7f258d2c4700 1 --2- 172.21.15.64:0/3066134853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 0x7f258810d760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f25780098f0 tx=0x7f2578009c00 comp rx=0 tx=0).stop 2024-05-08T22:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 -- 172.21.15.64:0/3066134853 shutdown_connections 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 --2- 172.21.15.64:0/3066134853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 0x7f258810d760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 --2- 172.21.15.64:0/3066134853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588107fa0 0x7f258810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 --2- 172.21.15.64:0/3066134853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f258810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 -- 172.21.15.64:0/3066134853 >> 172.21.15.64:0/3066134853 conn(0x7f258806df30 msgr2=0x7f2588070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 -- 172.21.15.64:0/3066134853 shutdown_connections 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.874+0000 7f258d2c4700 1 -- 172.21.15.64:0/3066134853 wait complete. 2024-05-08T22:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 Processor -- start 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 -- start start 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588107fa0 0x7f25881145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f2588114ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 0x7f25881b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588117b70 con 0x7f2588107fa0 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588117ce0 con 0x7f258810a910 2024-05-08T22:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.875+0000 7f258d2c4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25881b6330 con 0x7f258810d2e0 2024-05-08T22:04:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.883+0000 7f25867fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f2588114ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.883+0000 7f25867fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f2588114ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:59338/0 (socket says 172.21.15.64:59338) 2024-05-08T22:04:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.883+0000 7f25867fc700 1 -- 172.21.15.64:0/344469282 learned_addr learned my addr 172.21.15.64:0/344469282 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25877fe700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 0x7f25881b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f2586ffd700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588107fa0 0x7f25881145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25867fc700 1 -- 172.21.15.64:0/344469282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 msgr2=0x7f25881b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25867fc700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 0x7f25881b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25867fc700 1 -- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588107fa0 msgr2=0x7f25881145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25867fc700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588107fa0 0x7f25881145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25867fc700 1 -- 172.21.15.64:0/344469282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009520 con 0x7f258810a910 2024-05-08T22:04:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.884+0000 7f25867fc700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f2588114ae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e9d0 tx=0x7f257c00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.887+0000 7f256ffff700 1 -- 172.21.15.64:0/344469282 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00cb70 con 0x7f258810a910 2024-05-08T22:04:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.887+0000 7f256ffff700 1 -- 172.21.15.64:0/344469282 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257c00ccd0 con 0x7f258810a910 2024-05-08T22:04:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.887+0000 7f256ffff700 1 -- 172.21.15.64:0/344469282 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c018680 con 0x7f258810a910 2024-05-08T22:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.887+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881b6590 con 0x7f258810a910 2024-05-08T22:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.887+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881b6a50 con 0x7f258810a910 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:34 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:34 smithi186 ceph-mon[46619]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 759 op/s 2024-05-08T22:04:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.891+0000 7f259947eb80 1 Processor -- start 2024-05-08T22:04:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.891+0000 7f259947eb80 1 -- start start 2024-05-08T22:04:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f259947eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f259947eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f259947eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976b02d0 0x5648976b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f259947eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976060f0 con 0x5648976af3e0 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f259947eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564897603ab0 con 0x5648976b02d0 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f259947eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648975ff3a0 con 0x5648976aea80 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f256ffff700 1 -- 172.21.15.64:0/344469282 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f257c0187e0 con 0x7f258810a910 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.892+0000 7f256ffff700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25700a1320 0x7f25700a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f256ffff700 1 -- 172.21.15.64:0/344469282 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f257c014070 con 0x7f258810a910 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40992/0 (socket says 172.21.15.64:40992) 2024-05-08T22:04:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 -- 172.21.15.64:0/3632127891 learned_addr learned my addr 172.21.15.64:0/3632127891 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 -- 172.21.15.64:0/3632127891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 msgr2=0x5648976aeea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976aeea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 -- 172.21.15.64:0/3632127891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976b02d0 msgr2=0x5648976b0750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976b02d0 0x5648976b0750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.893+0000 7f2586ffd700 1 -- 172.21.15.64:0/3632127891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564897480350 con 0x5648976af3e0 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.894+0000 7f2586ffd700 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976afcf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2580009410 tx=0x7f25800099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a95dc005f22385 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.894+0000 7f256d7fa700 1 -- 172.21.15.64:0/3632127891 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000e070 con 0x5648976af3e0 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.894+0000 7f256d7fa700 1 -- 172.21.15.64:0/3632127891 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258000f040 con 0x5648976af3e0 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.894+0000 7f256d7fa700 1 -- 172.21.15.64:0/3632127891 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580015b50 con 0x5648976af3e0 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.895+0000 7f259947eb80 1 -- 172.21.15.64:0/3632127891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 msgr2=0x5648976afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.895+0000 7f259947eb80 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976afcf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2580009410 tx=0x7f25800099e0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.896+0000 7f25877fe700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25700a1320 0x7f25700a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.896+0000 7f259947eb80 1 -- 172.21.15.64:0/3632127891 shutdown_connections 2024-05-08T22:04:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.896+0000 7f259947eb80 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976b02d0 0x5648976b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.896+0000 7f259947eb80 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.896+0000 7f259947eb80 1 --2- 172.21.15.64:0/3632127891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976afcf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.896+0000 7f259947eb80 1 -- 172.21.15.64:0/3632127891 >> 172.21.15.64:0/3632127891 conn(0x56489760d1b0 msgr2=0x5648976ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.897+0000 7f25877fe700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25700a1320 0x7f25700a37e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2578009fd0 tx=0x7f257800fb10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.897+0000 7f259947eb80 1 -- 172.21.15.64:0/3632127891 shutdown_connections 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 -- 172.21.15.64:0/3632127891 wait complete. 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 -- start start 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 Processor -- start 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976aea80 0x5648976b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 0x5648976bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56489759b500 con 0x5648976af3e0 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56489759b1a0 con 0x5648976aea80 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564897482640 con 0x5648976b02d0 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.898+0000 7f259947eb80 10 client.0 _ll_register_callbacks cb 0x5648975a4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 0x5648976bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 0x5648976bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48714/0 (socket says 172.21.15.64:48714) 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 -- 172.21.15.64:0/4291835620 learned_addr learned my addr 172.21.15.64:0/4291835620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 -- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976aea80 msgr2=0x5648976b88e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976aea80 0x5648976b88e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 -- 172.21.15.64:0/4291835620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 msgr2=0x5648976b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.900+0000 7f25877fe700 1 -- 172.21.15.64:0/4291835620 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800090f0 con 0x5648976b02d0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f25877fe700 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 0x5648976bfe60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2578016000 tx=0x7f2578009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cec4f88c90277ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f256aff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f256aff5700 1 -- 172.21.15.64:0/4291835620 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257801d070 con 0x5648976b02d0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564897558210 con 0x5648976b02d0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f256aff5700 1 -- 172.21.15.64:0/4291835620 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2578005250 con 0x5648976b02d0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f256aff5700 1 -- 172.21.15.64:0/4291835620 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257800bc10 con 0x5648976b02d0 2024-05-08T22:04:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f259947eb80 10 client.25582 fetch_fsmap learned FSMap version 42 2024-05-08T22:04:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56489742cb00 con 0x5648976b02d0 2024-05-08T22:04:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.901+0000 7f256aff5700 1 -- 172.21.15.64:0/4291835620 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564897558210 con 0x5648976b02d0 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f256aff5700 1 -- 172.21.15.64:0/4291835620 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 42) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2578038070 con 0x5648976b02d0 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f259947eb80 10 client.25582 fetch_fsmap finished waiting for FSMap version 42 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f259947eb80 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f259947eb80 20 client.25582 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f259947eb80 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648976c47a0 0x5648976c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f259947eb80 4 client.25582 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56489767e7b0 con 0x5648976c47a0 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f2586ffd700 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648976c47a0 0x5648976c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f256aff5700 10 client.25582 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.903+0000 7f2586ffd700 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648976c47a0 0x5648976c6bc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739723e4c1104e02 server_cookie=3c6bedc4b1e051e in_seq=0 out_seq=0 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f256aff5700 1 -- 172.21.15.64:0/4291835620 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56489767e7b0 con 0x5648976c47a0 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f256aff5700 10 client.25582 handle_command_reply: tid=0 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 client.25582 shutdown 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 msgr2=0x5648976bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 0x5648976bfe60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2578016000 tx=0x7f2578009260 comp rx=0 tx=0).stop 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 shutdown_connections 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976aea80 0x5648976b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648976c47a0 0x5648976c6bc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976b02d0 0x5648976bfe60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 --2- 172.21.15.64:0/4291835620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976af3e0 0x5648976b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 >> 172.21.15.64:0/4291835620 conn(0x56489760d1b0 msgr2=0x5648976a9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.904+0000 7f256aff5700 0 client.25582 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.905+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 shutdown_connections 2024-05-08T22:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.905+0000 7f259947eb80 1 -- 172.21.15.64:0/4291835620 wait complete. 2024-05-08T22:04:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.905+0000 7f259947eb80 20 client.25582 trim_cache size 0 max 16384 2024-05-08T22:04:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 Processor -- start 2024-05-08T22:04:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 -- start start 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 0x5648976cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976b9560 con 0x5648976b02d0 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976b6480 con 0x5648976af3e0 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f259947eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976b3d70 con 0x5648976aea80 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f2586ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f25867fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f2586ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48724/0 (socket says 172.21.15.64:48724) 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.925+0000 7f2586ffd700 1 -- 172.21.15.64:0/791422966 learned_addr learned my addr 172.21.15.64:0/791422966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f25867fc700 1 -- 172.21.15.64:0/791422966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 msgr2=0x5648976b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f25867fc700 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f25867fc700 1 -- 172.21.15.64:0/791422966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 msgr2=0x5648976cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f25867fc700 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 0x5648976cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f25867fc700 1 -- 172.21.15.64:0/791422966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648975c5500 con 0x5648976af3e0 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f2586ffd700 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f25867fc700 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bfd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5648976afab0 tx=0x7f257c0ccf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383887482d91f16d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f256dffb700 1 -- 172.21.15.64:0/791422966 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c09aa00 con 0x5648976af3e0 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f256dffb700 1 -- 172.21.15.64:0/791422966 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257c09a300 con 0x5648976af3e0 2024-05-08T22:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.926+0000 7f256dffb700 1 -- 172.21.15.64:0/791422966 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0997f0 con 0x5648976af3e0 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 -- 172.21.15.64:0/791422966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 msgr2=0x5648976bfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bfd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5648976afab0 tx=0x7f257c0ccf50 comp rx=0 tx=0).stop 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 -- 172.21.15.64:0/791422966 shutdown_connections 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 0x5648976cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 --2- 172.21.15.64:0/791422966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bfd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.927+0000 7f259947eb80 1 -- 172.21.15.64:0/791422966 >> 172.21.15.64:0/791422966 conn(0x56489760d1b0 msgr2=0x5648976ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 -- 172.21.15.64:0/791422966 shutdown_connections 2024-05-08T22:04:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 -- 172.21.15.64:0/791422966 wait complete. 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 -- start start 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 Processor -- start 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976bd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 0x5648976be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976beae0 con 0x5648976b02d0 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976bec50 con 0x5648976af3e0 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648976bee10 con 0x5648976aea80 2024-05-08T22:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.930+0000 7f259947eb80 10 client.0 _ll_register_callbacks cb 0x5648975a4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:59388/0 (socket says 172.21.15.64:59388) 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 -- 172.21.15.64:0/3663109286 learned_addr learned my addr 172.21.15.64:0/3663109286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f2586ffd700 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 -- 172.21.15.64:0/3663109286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 msgr2=0x5648976bd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976bd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 -- 172.21.15.64:0/3663109286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 msgr2=0x5648976be5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 0x5648976be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 -- 172.21.15.64:0/3663109286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564897480350 con 0x5648976af3e0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f25867fc700 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bd590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f257c011fd0 tx=0x7f257c0987f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c1e4d13026d018 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f256aff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f256aff5700 1 -- 172.21.15.64:0/3663109286 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c09e130 con 0x5648976af3e0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f256aff5700 1 -- 172.21.15.64:0/3663109286 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f257c0a2780 con 0x5648976af3e0 2024-05-08T22:04:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.931+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f258000f040 con 0x5648976af3e0 2024-05-08T22:04:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.933+0000 7f256aff5700 1 -- 172.21.15.64:0/3663109286 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257c00ce40 con 0x5648976af3e0 2024-05-08T22:04:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.933+0000 7f259947eb80 10 client.25712 fetch_fsmap learned FSMap version 42 2024-05-08T22:04:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.933+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f25881b6a50 con 0x5648976af3e0 2024-05-08T22:04:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.933+0000 7f256aff5700 1 -- 172.21.15.64:0/3663109286 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258000f040 con 0x5648976af3e0 2024-05-08T22:04:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.933+0000 7f256aff5700 1 -- 172.21.15.64:0/3663109286 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 42) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f257c018a90 con 0x5648976af3e0 2024-05-08T22:04:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.934+0000 7f259947eb80 10 client.25712 fetch_fsmap finished waiting for FSMap version 42 2024-05-08T22:04:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.934+0000 7f259947eb80 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f259947eb80 20 client.25712 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f259947eb80 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56489771aed0 0x56489771d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f259947eb80 4 client.25712 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648975f7e60 con 0x56489771aed0 2024-05-08T22:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f2586ffd700 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56489771aed0 0x56489771d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f256aff5700 10 client.25712 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.935+0000 7f2586ffd700 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56489771aed0 0x56489771d2f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8698dcbb215dfdb server_cookie=de2de94537a772fb in_seq=0 out_seq=0 2024-05-08T22:04:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.944+0000 7f256aff5700 1 -- 172.21.15.64:0/3663109286 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648975f7e60 con 0x56489771aed0 2024-05-08T22:04:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.944+0000 7f256aff5700 10 client.25712 handle_command_reply: tid=0 2024-05-08T22:04:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.944+0000 7f259947eb80 1 client.25712 shutdown 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.944+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 msgr2=0x5648976bd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.944+0000 7f259947eb80 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bd590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f257c011fd0 tx=0x7f257c0987f0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.944+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 shutdown_connections 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56489771aed0 0x56489771d2f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648976b02d0 0x5648976be5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648976aea80 0x5648976bd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 --2- 172.21.15.64:0/3663109286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648976af3e0 0x5648976bd590 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 >> 172.21.15.64:0/3663109286 conn(0x56489760d1b0 msgr2=0x5648976cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f256aff5700 0 client.25712 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 shutdown_connections 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 1 -- 172.21.15.64:0/3663109286 wait complete. 2024-05-08T22:04:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.945+0000 7f259947eb80 20 client.25712 trim_cache size 0 max 16384 2024-05-08T22:04:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:04:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:04:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:04:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:04:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25700a1320 msgr2=0x7f25700a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25700a1320 0x7f25700a37e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2578009fd0 tx=0x7f257800fb10 comp rx=0 tx=0).stop 2024-05-08T22:04:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 msgr2=0x7f2588114ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f2588114ae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e9d0 tx=0x7f257c00ed90 comp rx=0 tx=0).stop 2024-05-08T22:04:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 shutdown_connections 2024-05-08T22:04:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f25700a1320 0x7f25700a37e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f258810d2e0 0x7f25881b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588107fa0 0x7f25881145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 --2- 172.21.15.64:0/344469282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258810a910 0x7f2588114ae0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 >> 172.21.15.64:0/344469282 conn(0x7f258806df30 msgr2=0x7f258810bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 shutdown_connections 2024-05-08T22:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:34.946+0000 7f258d2c4700 1 -- 172.21.15.64:0/344469282 wait complete. 2024-05-08T22:04:34.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:34 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:34.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:34 smithi064 ceph-mon[42500]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 759 op/s 2024-05-08T22:04:35.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:35.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:35.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:04:35.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:35.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:35.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:04:35.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:35.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:35.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:04:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 585.0 s, 1653.2 tps, lat 59.183 ms stddev 68.781 2024-05-08T22:04:36.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:36 smithi186 ceph-mon[46619]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 697 op/s 2024-05-08T22:04:36.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:36 smithi064 ceph-mon[42500]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 697 op/s 2024-05-08T22:04:37.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:36 smithi160 ceph-mon[46962]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 697 op/s 2024-05-08T22:04:38.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:38 smithi160 ceph-mon[46962]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 MiB/s wr, 635 op/s 2024-05-08T22:04:38.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:38 smithi186 ceph-mon[46619]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 MiB/s wr, 635 op/s 2024-05-08T22:04:38.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:38 smithi064 ceph-mon[42500]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 MiB/s wr, 635 op/s 2024-05-08T22:04:40.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:40 smithi160 ceph-mon[46962]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-05-08T22:04:40.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:40 smithi186 ceph-mon[46619]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-05-08T22:04:40.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:40 smithi064 ceph-mon[42500]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-05-08T22:04:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 590.0 s, 1051.5 tps, lat 95.858 ms stddev 118.031 2024-05-08T22:04:42.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:42 smithi160 ceph-mon[46962]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-08T22:04:42.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:42 smithi186 ceph-mon[46619]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-08T22:04:42.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:42 smithi064 ceph-mon[42500]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-08T22:04:44.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:44 smithi160 ceph-mon[46962]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 854 op/s 2024-05-08T22:04:44.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:44 smithi186 ceph-mon[46619]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 854 op/s 2024-05-08T22:04:44.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:44 smithi064 ceph-mon[42500]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 854 op/s 2024-05-08T22:04:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 595.0 s, 1610.8 tps, lat 62.861 ms stddev 76.746 2024-05-08T22:04:46.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:46 smithi186 ceph-mon[46619]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 789 op/s 2024-05-08T22:04:46.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:46 smithi064 ceph-mon[42500]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 789 op/s 2024-05-08T22:04:47.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:46 smithi160 ceph-mon[46962]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 789 op/s 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 -- 172.21.15.64:0/1687446638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c100770 msgr2=0x7f4d6c100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 --2- 172.21.15.64:0/1687446638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c100770 0x7f4d6c100bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d6001aa20 tx=0x7f4d6001ad30 comp rx=0 tx=0).stop 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 -- 172.21.15.64:0/1687446638 shutdown_connections 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 --2- 172.21.15.64:0/1687446638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c101110 0x7f4d6c1079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 --2- 172.21.15.64:0/1687446638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 --2- 172.21.15.64:0/1687446638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c100770 0x7f4d6c100bd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 -- 172.21.15.64:0/1687446638 >> 172.21.15.64:0/1687446638 conn(0x7f4d6c0faaf0 msgr2=0x7f4d6c0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 -- 172.21.15.64:0/1687446638 shutdown_connections 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.113+0000 7f4d7158b700 1 -- 172.21.15.64:0/1687446638 wait complete. 2024-05-08T22:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 Processor -- start 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 -- start start 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c1070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c100770 0x7f4d6c1075f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c101110 0x7f4d6c10ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c10f240 con 0x7f4d6c100770 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c10f3b0 con 0x7f4d6c0ff580 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.114+0000 7f4d7158b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c10f570 con 0x7f4d6c101110 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c1070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c1070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51590/0 (socket says 172.21.15.64:51590) 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 -- 172.21.15.64:0/3897444431 learned_addr learned my addr 172.21.15.64:0/3897444431 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 -- 172.21.15.64:0/3897444431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c101110 msgr2=0x7f4d6c10ec70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c101110 0x7f4d6c10ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 -- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c100770 msgr2=0x7f4d6c1075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c100770 0x7f4d6c1075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.115+0000 7f4d6affd700 1 -- 172.21.15.64:0/3897444431 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6001a600 con 0x7f4d6c0ff580 2024-05-08T22:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.118+0000 7f4d6affd700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c1070b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c050930 tx=0x7f4d5c050cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.118+0000 7f4d53fff700 1 -- 172.21.15.64:0/3897444431 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c004020 con 0x7f4d6c0ff580 2024-05-08T22:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.118+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c10f7f0 con 0x7f4d6c0ff580 2024-05-08T22:04:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.118+0000 7f4d53fff700 1 -- 172.21.15.64:0/3897444431 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5c04b910 con 0x7f4d6c0ff580 2024-05-08T22:04:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.118+0000 7f4d53fff700 1 -- 172.21.15.64:0/3897444431 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c003b20 con 0x7f4d6c0ff580 2024-05-08T22:04:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.118+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c10fd50 con 0x7f4d6c0ff580 2024-05-08T22:04:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 Processor -- start 2024-05-08T22:04:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d53fff700 1 -- 172.21.15.64:0/3897444431 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4d5c04ba80 con 0x7f4d6c0ff580 2024-05-08T22:04:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 -- start start 2024-05-08T22:04:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe0a80 0x55db1efe0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1efe1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1efe2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1ef380f0 con 0x55db1efe13e0 2024-05-08T22:04:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1ef35ab0 con 0x55db1efe22d0 2024-05-08T22:04:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1ef313a0 con 0x55db1efe0a80 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.123+0000 7f4d6a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1efe2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1efe2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51602/0 (socket says 172.21.15.64:51602) 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2260082263 learned_addr learned my addr 172.21.15.64:0/2260082263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d53fff700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4d540a1180 0x7f4d540a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe0a80 0x55db1efe0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2260082263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe0a80 msgr2=0x55db1efe0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe0a80 0x55db1efe0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2260082263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 msgr2=0x55db1efe1cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:04:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1efe1cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2260082263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db1edb2350 con 0x55db1efe22d0 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4d540a1180 0x7f4d540a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1efe2750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d6001a2f0 tx=0x7f4d6001a930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ba7a973ca50d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.124+0000 7f4d517fa700 1 -- 172.21.15.64:0/2260082263 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60003930 con 0x55db1efe22d0 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.125+0000 7f4d517fa700 1 -- 172.21.15.64:0/2260082263 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d60004660 con 0x55db1efe22d0 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.125+0000 7f4d517fa700 1 -- 172.21.15.64:0/2260082263 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60030470 con 0x55db1efe22d0 2024-05-08T22:04:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.125+0000 7f4d53fff700 1 -- 172.21.15.64:0/3897444431 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4d5c0dd330 con 0x7f4d6c0ff580 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.125+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2260082263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 msgr2=0x55db1efe2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.125+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4d540a1180 0x7f4d540a3640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d640073e0 tx=0x7f4d6400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.125+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1efe2750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d6001a2f0 tx=0x7f4d6001a930 comp rx=0 tx=0).stop 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.126+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2260082263 shutdown_connections 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.126+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1efe2750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.126+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe0a80 0x55db1efe0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.126+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2260082263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1efe1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.126+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2260082263 >> 172.21.15.64:0/2260082263 conn(0x55db1ef3f1b0 msgr2=0x55db1efdd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.128+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2260082263 shutdown_connections 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.128+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2260082263 wait complete. 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.128+0000 7f4d7d745b80 1 -- start start 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 Processor -- start 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe13e0 0x55db1efeadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1eff1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1eecd500 con 0x55db1efe0a80 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1eecd1a0 con 0x55db1efe22d0 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1edb4640 con 0x55db1efe13e0 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d7d745b80 10 client.0 _ll_register_callbacks cb 0x55db1eed6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d6b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1eff1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d6b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1eff1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51604/0 (socket says 172.21.15.64:51604) 2024-05-08T22:04:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/402255885 learned_addr learned my addr 172.21.15.64:0/402255885 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d6affd700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.129+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/402255885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe13e0 msgr2=0x55db1efeadd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe13e0 0x55db1efeadd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/402255885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 msgr2=0x55db1efea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/402255885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c108310 con 0x55db1efe22d0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d6affd700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1eff1e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d64008a70 tx=0x7f4d64004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1992ae0174815 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d4eff5700 1 -- 172.21.15.64:0/402255885 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400f040 con 0x55db1efe22d0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d4eff5700 1 -- 172.21.15.64:0/402255885 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4d64014880 con 0x55db1efe22d0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db1ee8a210 con 0x55db1efe22d0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d4eff5700 1 -- 172.21.15.64:0/402255885 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d64007ea0 con 0x55db1efe22d0 2024-05-08T22:04:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d7d745b80 10 client.25724 fetch_fsmap learned FSMap version 42 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.130+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55db1ed5eb00 con 0x55db1efe22d0 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d4eff5700 1 -- 172.21.15.64:0/402255885 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db1ee8a210 con 0x55db1efe22d0 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d4eff5700 1 -- 172.21.15.64:0/402255885 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 42) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4d64028070 con 0x55db1efe22d0 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d7d745b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 42 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d7d745b80 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d7d745b80 20 client.25724 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1eff6750 0x55db1eff8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d7d745b80 4 client.25724 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db1efb07b0 con 0x55db1eff6750 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d6affd700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1eff6750 0x55db1eff8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d4eff5700 10 client.25724 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.131+0000 7f4d6affd700 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1eff6750 0x55db1eff8b70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7524b8bfa7639e2 server_cookie=49cb8ec6fee86cd1 in_seq=0 out_seq=0 2024-05-08T22:04:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d4eff5700 1 -- 172.21.15.64:0/402255885 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db1efb07b0 con 0x55db1eff6750 2024-05-08T22:04:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d4eff5700 10 client.25724 handle_command_reply: tid=0 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 client.25724 shutdown 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 msgr2=0x55db1eff1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1eff1e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d64008a70 tx=0x7f4d64004b40 comp rx=0 tx=0).stop 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 shutdown_connections 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe22d0 0x55db1eff1e10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1eff6750 0x55db1eff8b70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/402255885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe13e0 0x55db1efeadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d4eff5700 0 client.25724 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 >> 172.21.15.64:0/402255885 conn(0x55db1ef3f1b0 msgr2=0x55db1efdb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 shutdown_connections 2024-05-08T22:04:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 1 -- 172.21.15.64:0/402255885 wait complete. 2024-05-08T22:04:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.132+0000 7f4d7d745b80 20 client.25724 trim_cache size 0 max 16384 2024-05-08T22:04:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 Processor -- start 2024-05-08T22:04:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 -- start start 2024-05-08T22:04:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 0x55db1eff24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1eff1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1efe5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1efeb8c0 con 0x55db1efe13e0 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1efdc930 con 0x55db1efe0a80 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.158+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1eff3b70 con 0x55db1efe22d0 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 0x55db1eff24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1efe5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 0x55db1eff24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51612/0 (socket says 172.21.15.64:51612) 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6affd700 1 -- 172.21.15.64:0/2269119919 learned_addr learned my addr 172.21.15.64:0/2269119919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1efe5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53900/0 (socket says 172.21.15.64:53900) 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1eff1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/2269119919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 msgr2=0x55db1eff24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 0x55db1eff24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/2269119919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 msgr2=0x55db1eff1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1eff1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 -- 172.21.15.64:0/2269119919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db1eef7500 con 0x55db1efe22d0 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6affd700 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 0x55db1eff24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1efe5510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400adb0 tx=0x7f4d64008ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d3a9bdbf0cb9f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d51ffb700 1 -- 172.21.15.64:0/2269119919 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400f040 con 0x55db1efe22d0 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d51ffb700 1 -- 172.21.15.64:0/2269119919 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d640208c0 con 0x55db1efe22d0 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.159+0000 7f4d51ffb700 1 -- 172.21.15.64:0/2269119919 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64014bb0 con 0x55db1efe22d0 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2269119919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 msgr2=0x55db1efe5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1efe5510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400adb0 tx=0x7f4d64008ee0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2269119919 shutdown_connections 2024-05-08T22:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe0a80 0x55db1eff24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1efe5510 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/2269119919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe13e0 0x55db1eff1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2269119919 >> 172.21.15.64:0/2269119919 conn(0x55db1ef3f1b0 msgr2=0x55db1efe8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2269119919 shutdown_connections 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 -- 172.21.15.64:0/2269119919 wait complete. 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.160+0000 7f4d7d745b80 1 -- start start 2024-05-08T22:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 Processor -- start 2024-05-08T22:04:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe13e0 0x55db1efefc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1eff0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1eff1150 con 0x55db1efe0a80 2024-05-08T22:04:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1eff12c0 con 0x55db1efe13e0 2024-05-08T22:04:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1eff1480 con 0x55db1efe22d0 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d7d745b80 10 client.0 _ll_register_callbacks cb 0x55db1eed6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efef6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52566/0 (socket says 172.21.15.64:52566) 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 -- 172.21.15.64:0/4212355258 learned_addr learned my addr 172.21.15.64:0/4212355258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe13e0 0x55db1efefc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1eff0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 -- 172.21.15.64:0/4212355258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 msgr2=0x55db1eff0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1eff0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 -- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe13e0 msgr2=0x55db1efefc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe13e0 0x55db1efefc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 -- 172.21.15.64:0/4212355258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c10fd50 con 0x55db1efe0a80 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe13e0 0x55db1efefc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.161+0000 7f4d6affd700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efef6c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0d9130 tx=0x7f4d5c050f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2672993ac189e525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d4eff5700 1 -- 172.21.15.64:0/4212355258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0e50b0 con 0x55db1efe0a80 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d4eff5700 1 -- 172.21.15.64:0/4212355258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4d5c003c80 con 0x55db1efe0a80 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d60004660 con 0x55db1efe0a80 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d4eff5700 1 -- 172.21.15.64:0/4212355258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d5c003e40 con 0x55db1efe0a80 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 10 client.15945 fetch_fsmap learned FSMap version 42 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55db1ed5eb00 con 0x55db1efe0a80 2024-05-08T22:04:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d4eff5700 1 -- 172.21.15.64:0/4212355258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5c0fb030 con 0x55db1efe0a80 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d4eff5700 1 -- 172.21.15.64:0/4212355258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 42) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4d5c0e4c20 con 0x55db1efe0a80 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 42 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 20 client.15945 populate_metadata read hostname 'smithi064' 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1f04d8a0 0x55db1f04fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 4 client.15945 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:04:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.162+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db1ef29e60 con 0x55db1f04d8a0 2024-05-08T22:04:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.163+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1f04d8a0 0x55db1f04fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.163+0000 7f4d6a7fc700 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1f04d8a0 0x55db1f04fcc0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de84ac3836a97e server_cookie=3aeca42320665c81 in_seq=0 out_seq=0 2024-05-08T22:04:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.163+0000 7f4d4eff5700 10 client.15945 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.170+0000 7f4d4eff5700 1 -- 172.21.15.64:0/4212355258 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db1ef29e60 con 0x55db1f04d8a0 2024-05-08T22:04:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.170+0000 7f4d4eff5700 10 client.15945 handle_command_reply: tid=0 2024-05-08T22:04:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.170+0000 7f4d7d745b80 1 client.15945 shutdown 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 msgr2=0x55db1efef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efef6c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0d9130 tx=0x7f4d5c050f30 comp rx=0 tx=0).stop 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 shutdown_connections 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55db1f04d8a0 0x55db1f04fcc0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db1efe22d0 0x55db1eff0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db1efe0a80 0x55db1efef6c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 --2- 172.21.15.64:0/4212355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db1efe13e0 0x55db1efefc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d4eff5700 0 client.15945 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:04:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 >> 172.21.15.64:0/4212355258 conn(0x55db1ef3f1b0 msgr2=0x55db1efe9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 shutdown_connections 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 1 -- 172.21.15.64:0/4212355258 wait complete. 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7d745b80 20 client.15945 trim_cache size 0 max 16384 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:04:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4d540a1180 msgr2=0x7f4d540a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4d540a1180 0x7f4d540a3640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d640073e0 tx=0x7f4d6400c040 comp rx=0 tx=0).stop 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 msgr2=0x7f4d6c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c1070b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c050930 tx=0x7f4d5c050cf0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 shutdown_connections 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f4d540a1180 0x7f4d540a3640 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d6c0ff580 0x7f4d6c1070b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6c101110 0x7f4d6c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 --2- 172.21.15.64:0/3897444431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c100770 0x7f4d6c1075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.171+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 >> 172.21.15.64:0/3897444431 conn(0x7f4d6c0faaf0 msgr2=0x7f4d6c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.172+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 shutdown_connections 2024-05-08T22:04:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:04:48.172+0000 7f4d7158b700 1 -- 172.21.15.64:0/3897444431 wait complete. 2024-05-08T22:04:48.794 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:48 smithi160 ceph-mon[46962]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 788 op/s 2024-05-08T22:04:48.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:48 smithi186 ceph-mon[46619]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 788 op/s 2024-05-08T22:04:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:48 smithi064 ceph-mon[42500]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 788 op/s 2024-05-08T22:04:51.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 600.0 s, 1601.4 tps, lat 62.541 ms stddev 67.192 2024-05-08T22:04:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:51 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:51.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:51 smithi064 ceph-mon[42500]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 886 op/s 2024-05-08T22:04:51.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:51 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:51.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:51 smithi160 ceph-mon[46962]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 886 op/s 2024-05-08T22:04:51.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:51 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:51.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:51 smithi186 ceph-mon[46619]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 886 op/s 2024-05-08T22:04:52.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:52 smithi186 ceph-mon[46619]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T22:04:52.970 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:52 smithi064 ceph-mon[42500]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T22:04:53.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:52 smithi160 ceph-mon[46962]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T22:04:54.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:54 smithi186 ceph-mon[46619]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-05-08T22:04:54.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:54 smithi064 ceph-mon[42500]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-05-08T22:04:55.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:54 smithi160 ceph-mon[46962]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-05-08T22:04:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 605.0 s, 1471.2 tps, lat 68.121 ms stddev 75.365 2024-05-08T22:04:56.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:56 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:56.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:56 smithi186 ceph-mon[46619]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 687 op/s 2024-05-08T22:04:56.974 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:56 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:56.974 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:56 smithi064 ceph-mon[42500]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 687 op/s 2024-05-08T22:04:57.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:56 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:04:57.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:56 smithi160 ceph-mon[46962]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 687 op/s 2024-05-08T22:04:58.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:04:58 smithi186 ceph-mon[46619]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 608 op/s 2024-05-08T22:04:58.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:04:58 smithi064 ceph-mon[42500]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 608 op/s 2024-05-08T22:04:59.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:04:58 smithi160 ceph-mon[46962]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 608 op/s 2024-05-08T22:05:00.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:00 smithi186 ceph-mon[46619]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 MiB/s wr, 1.52k op/s 2024-05-08T22:05:00.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:00 smithi064 ceph-mon[42500]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 MiB/s wr, 1.52k op/s 2024-05-08T22:05:01.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:00 smithi160 ceph-mon[46962]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 MiB/s wr, 1.52k op/s 2024-05-08T22:05:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.346+0000 7f34b040f700 1 -- 172.21.15.64:0/2439296056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a8101990 msgr2=0x7f34a8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.346+0000 7f34b040f700 1 --2- 172.21.15.64:0/2439296056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a8101990 0x7f34a8108240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009fc0 tx=0x7f34a400b330 comp rx=0 tx=0).stop 2024-05-08T22:05:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 -- 172.21.15.64:0/2439296056 shutdown_connections 2024-05-08T22:05:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 --2- 172.21.15.64:0/2439296056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a8101990 0x7f34a8108240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 --2- 172.21.15.64:0/2439296056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a80ffe00 0x7f34a8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 --2- 172.21.15.64:0/2439296056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 -- 172.21.15.64:0/2439296056 >> 172.21.15.64:0/2439296056 conn(0x7f34a80fb390 msgr2=0x7f34a80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 -- 172.21.15.64:0/2439296056 shutdown_connections 2024-05-08T22:05:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 -- 172.21.15.64:0/2439296056 wait complete. 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 Processor -- start 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.347+0000 7f34b040f700 1 -- start start 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34b040f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 0x7f34a8194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34b040f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34b040f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a8101990 0x7f34a819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34b040f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a819c950 con 0x7f34a8101990 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34b040f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a819cac0 con 0x7f34a80ffe00 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34b040f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a819cc80 con 0x7f34a8100ff0 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ad9aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae1ab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 0x7f34a8194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a8101990 0x7f34a819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ad9aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:54138/0 (socket says 172.21.15.64:54138) 2024-05-08T22:05:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ad9aa700 1 -- 172.21.15.64:0/3794390131 learned_addr learned my addr 172.21.15.64:0/3794390131 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae1ab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 0x7f34a8194e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47918/0 (socket says 172.21.15.64:47918) 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 -- 172.21.15.64:0/3794390131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 msgr2=0x7f34a8195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8195360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 -- 172.21.15.64:0/3794390131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 msgr2=0x7f34a8194e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 0x7f34a8194e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 -- 172.21.15.64:0/3794390131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00095f0 con 0x7f34a8101990 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 0x7f34a8194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.348+0000 7f34ae9ac700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a8101990 0x7f34a819c380 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f34a400b6c0 tx=0x7f34a4007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.349+0000 7f349f7fe700 1 -- 172.21.15.64:0/3794390131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400e040 con 0x7f34a8101990 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.349+0000 7f349f7fe700 1 -- 172.21.15.64:0/3794390131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34a401d470 con 0x7f34a8101990 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.349+0000 7f349f7fe700 1 -- 172.21.15.64:0/3794390131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40052a0 con 0x7f34a8101990 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.349+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a4009c40 con 0x7f34a8101990 2024-05-08T22:05:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.349+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a819d140 con 0x7f34a8101990 2024-05-08T22:05:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.353+0000 7f34bc5c9b80 1 Processor -- start 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.353+0000 7f34bc5c9b80 1 -- start start 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd32fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd32fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd32540f0 con 0x55cfd32fca80 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd3251ab0 con 0x55cfd32fe2d0 2024-05-08T22:05:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd324d3a0 con 0x55cfd32fd3e0 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47934/0 (socket says 172.21.15.64:47934) 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 -- 172.21.15.64:0/421665046 learned_addr learned my addr 172.21.15.64:0/421665046 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd32fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 -- 172.21.15.64:0/421665046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 msgr2=0x55cfd32fdcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd32fdcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 -- 172.21.15.64:0/421665046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 msgr2=0x55cfd32fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd32fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.354+0000 7f34ad9aa700 1 -- 172.21.15.64:0/421665046 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd30ce350 con 0x55cfd32fe2d0 2024-05-08T22:05:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.369+0000 7f349f7fe700 1 -- 172.21.15.64:0/3794390131 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34a401d850 con 0x7f34a8101990 2024-05-08T22:05:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.369+0000 7f349f7fe700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34940a12f0 0x7f34940a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.370+0000 7f349f7fe700 1 -- 172.21.15.64:0/3794390131 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34a40cd900 con 0x7f34a8101990 2024-05-08T22:05:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.377+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3498011d80 tx=0x7f3498009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feeee09c6e00da5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.380+0000 7f349cda4700 1 -- 172.21.15.64:0/421665046 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498011030 con 0x55cfd32fe2d0 2024-05-08T22:05:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.380+0000 7f349cda4700 1 -- 172.21.15.64:0/421665046 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f349800fe70 con 0x55cfd32fe2d0 2024-05-08T22:05:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.380+0000 7f349cda4700 1 -- 172.21.15.64:0/421665046 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498010ab0 con 0x55cfd32fe2d0 2024-05-08T22:05:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.387+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34940a12f0 0x7f34940a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.388+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/421665046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 msgr2=0x55cfd32fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.388+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34940a12f0 0x7f34940a37b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f34a8100e50 tx=0x7f34a0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.389+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3498011d80 tx=0x7f3498009d20 comp rx=0 tx=0).stop 2024-05-08T22:05:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.392+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/421665046 shutdown_connections 2024-05-08T22:05:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.392+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.392+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd32fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.392+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/421665046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd32fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.392+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/421665046 >> 172.21.15.64:0/421665046 conn(0x55cfd325b1b0 msgr2=0x55cfd32f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.395+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/421665046 shutdown_connections 2024-05-08T22:05:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.400+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/421665046 wait complete. 2024-05-08T22:05:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.400+0000 7f34bc5c9b80 1 -- start start 2024-05-08T22:05:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.400+0000 7f34bc5c9b80 1 Processor -- start 2024-05-08T22:05:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd3317960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fd3e0 0x55cfd3317ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 0x55cfd331eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd31e9500 con 0x55cfd32fca80 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd31e91a0 con 0x55cfd32fd3e0 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd30d0640 con 0x55cfd32fe2d0 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34bc5c9b80 10 client.0 _ll_register_callbacks cb 0x55cfd31f2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 0x55cfd331eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 0x55cfd331eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:54164/0 (socket says 172.21.15.64:54164) 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.403+0000 7f34ae9ac700 1 -- 172.21.15.64:0/778976072 learned_addr learned my addr 172.21.15.64:0/778976072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34ae9ac700 1 -- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fd3e0 msgr2=0x55cfd3317ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:05:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34ae9ac700 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fd3e0 0x55cfd3317ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34ae9ac700 1 -- 172.21.15.64:0/778976072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 msgr2=0x55cfd3317960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34ae9ac700 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd3317960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34ae9ac700 1 -- 172.21.15.64:0/778976072 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498019040 con 0x55cfd32fe2d0 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34ae9ac700 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 0x55cfd331eee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f34a409ad20 tx=0x7f34a4097200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dbe4503b652c862 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f3491ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f3491ffb700 1 -- 172.21.15.64:0/778976072 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a409a480 con 0x55cfd32fe2d0 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f3491ffb700 1 -- 172.21.15.64:0/778976072 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34a409ad90 con 0x55cfd32fe2d0 2024-05-08T22:05:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.404+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfd31a6210 con 0x55cfd32fe2d0 2024-05-08T22:05:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.407+0000 7f3491ffb700 1 -- 172.21.15.64:0/778976072 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34a409ebc0 con 0x55cfd32fe2d0 2024-05-08T22:05:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.407+0000 7f34bc5c9b80 10 client.25606 fetch_fsmap learned FSMap version 44 2024-05-08T22:05:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.407+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cfd307ab00 con 0x55cfd32fe2d0 2024-05-08T22:05:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.408+0000 7f3491ffb700 1 -- 172.21.15.64:0/778976072 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cfd31a6210 con 0x55cfd32fe2d0 2024-05-08T22:05:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f3491ffb700 1 -- 172.21.15.64:0/778976072 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f34a40a2cf0 con 0x55cfd32fe2d0 2024-05-08T22:05:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f34bc5c9b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T22:05:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f34bc5c9b80 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f34bc5c9b80 20 client.25606 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33235b0 0x55cfd3325a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f34bc5c9b80 4 client.25606 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:05:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.409+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfd32cc7b0 con 0x55cfd33235b0 2024-05-08T22:05:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.410+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33235b0 0x55cfd3325a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f3491ffb700 10 client.25606 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33235b0 0x55cfd3325a70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=818ac9bf776030d server_cookie=a37d31ebd2bea576 in_seq=0 out_seq=0 2024-05-08T22:05:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f3491ffb700 1 -- 172.21.15.64:0/778976072 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfd32cc7b0 con 0x55cfd33235b0 2024-05-08T22:05:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f3491ffb700 10 client.25606 handle_command_reply: tid=0 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f34bc5c9b80 1 client.25606 shutdown 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 msgr2=0x55cfd331eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.411+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 0x55cfd331eee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f34a409ad20 tx=0x7f34a4097200 comp rx=0 tx=0).stop 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 shutdown_connections 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33235b0 0x55cfd3325a70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fe2d0 0x55cfd331eee0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd3317960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f3491ffb700 0 client.25606 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/778976072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fd3e0 0x55cfd3317ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 >> 172.21.15.64:0/778976072 conn(0x55cfd325b1b0 msgr2=0x55cfd32f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 shutdown_connections 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/778976072 wait complete. 2024-05-08T22:05:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.412+0000 7f34bc5c9b80 20 client.25606 trim_cache size 0 max 16384 2024-05-08T22:05:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 Processor -- start 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 -- start start 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd33259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd32f49b0 con 0x55cfd32fca80 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd33163a0 con 0x55cfd32fe2d0 2024-05-08T22:05:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd32f8930 con 0x55cfd32fd3e0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47946/0 (socket says 172.21.15.64:47946) 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.433+0000 7f34ae9ac700 1 -- 172.21.15.64:0/665136202 learned_addr learned my addr 172.21.15.64:0/665136202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd33259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 -- 172.21.15.64:0/665136202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 msgr2=0x55cfd32fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 -- 172.21.15.64:0/665136202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 msgr2=0x55cfd33259d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd33259d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 -- 172.21.15.64:0/665136202 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd3213500 con 0x55cfd32fd3e0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331eea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55cfd32f23f0 tx=0x7f3498011640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9875aa30481e78b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.435+0000 7f349d5a5700 1 -- 172.21.15.64:0/665136202 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498024070 con 0x55cfd32fd3e0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.436+0000 7f349d5a5700 1 -- 172.21.15.64:0/665136202 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3498010430 con 0x55cfd32fd3e0 2024-05-08T22:05:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.436+0000 7f349d5a5700 1 -- 172.21.15.64:0/665136202 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34980046d0 con 0x55cfd32fd3e0 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.436+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/665136202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 msgr2=0x55cfd331eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.436+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331eea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55cfd32f23f0 tx=0x7f3498011640 comp rx=0 tx=0).stop 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/665136202 shutdown_connections 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd32fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd33259d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/665136202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331eea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/665136202 >> 172.21.15.64:0/665136202 conn(0x55cfd325b1b0 msgr2=0x55cfd3304890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/665136202 shutdown_connections 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.437+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/665136202 wait complete. 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 -- start start 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 Processor -- start 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd331be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd3377b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd33780b0 con 0x55cfd32fca80 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd33781f0 con 0x55cfd32fe2d0 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd3378360 con 0x55cfd32fd3e0 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34bc5c9b80 10 client.0 _ll_register_callbacks cb 0x55cfd31f2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ae1ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd331be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ad9aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd3377b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ae1ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd331be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60706/0 (socket says 172.21.15.64:60706) 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ae9ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd3377b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:47954/0 (socket says 172.21.15.64:47954) 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ad9aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:54178/0 (socket says 172.21.15.64:54178) 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.438+0000 7f34ae1ab700 1 -- 172.21.15.64:0/1513074161 learned_addr learned my addr 172.21.15.64:0/1513074161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ad9aa700 1 -- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 msgr2=0x55cfd3377b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd3377b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ad9aa700 1 -- 172.21.15.64:0/1513074161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 msgr2=0x55cfd331be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd331be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ad9aa700 1 -- 172.21.15.64:0/1513074161 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd30ce350 con 0x55cfd32fd3e0 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd331be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34ad9aa700 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331c370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f349800a010 tx=0x7f3498009db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d9292c0eda3584 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f3491ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f3491ffb700 1 -- 172.21.15.64:0/1513074161 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498024070 con 0x55cfd32fd3e0 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f3491ffb700 1 -- 172.21.15.64:0/1513074161 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3498000e40 con 0x55cfd32fd3e0 2024-05-08T22:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f349800fe70 con 0x55cfd32fd3e0 2024-05-08T22:05:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f3491ffb700 1 -- 172.21.15.64:0/1513074161 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f349800f980 con 0x55cfd32fd3e0 2024-05-08T22:05:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34bc5c9b80 10 client.25612 fetch_fsmap learned FSMap version 44 2024-05-08T22:05:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cfd30d2c70 con 0x55cfd32fd3e0 2024-05-08T22:05:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.439+0000 7f3491ffb700 1 -- 172.21.15.64:0/1513074161 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f349800fe70 con 0x55cfd32fd3e0 2024-05-08T22:05:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.440+0000 7f3491ffb700 1 -- 172.21.15.64:0/1513074161 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f349800fae0 con 0x55cfd32fd3e0 2024-05-08T22:05:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.441+0000 7f34bc5c9b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.441+0000 7f34bc5c9b80 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.441+0000 7f34bc5c9b80 20 client.25612 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.441+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33798d0 0x55cfd337bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.441+0000 7f34bc5c9b80 4 client.25612 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.441+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cfd3245e60 con 0x55cfd33798d0 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.442+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33798d0 0x55cfd337bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.442+0000 7f34ae1ab700 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33798d0 0x55cfd337bcc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b4f400972d1a3c server_cookie=70693d3758c0d418 in_seq=0 out_seq=0 2024-05-08T22:05:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.442+0000 7f3491ffb700 10 client.25612 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f3491ffb700 1 -- 172.21.15.64:0/1513074161 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cfd3245e60 con 0x55cfd33798d0 2024-05-08T22:05:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f3491ffb700 10 client.25612 handle_command_reply: tid=0 2024-05-08T22:05:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 client.25612 shutdown 2024-05-08T22:05:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 msgr2=0x55cfd331c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331c370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f349800a010 tx=0x7f3498009db0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 shutdown_connections 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfd32fe2d0 0x55cfd3377b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfd33798d0 0x55cfd337bcc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfd32fca80 0x55cfd331be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 --2- 172.21.15.64:0/1513074161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfd32fd3e0 0x55cfd331c370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f3491ffb700 0 client.25612 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.457+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 >> 172.21.15.64:0/1513074161 conn(0x55cfd325b1b0 msgr2=0x55cfd332a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.459+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 shutdown_connections 2024-05-08T22:05:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.459+0000 7f34bc5c9b80 1 -- 172.21.15.64:0/1513074161 wait complete. 2024-05-08T22:05:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.459+0000 7f34bc5c9b80 20 client.25612 trim_cache size 0 max 16384 2024-05-08T22:05:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:05:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:05:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:05:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:05:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34940a12f0 msgr2=0x7f34940a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34940a12f0 0x7f34940a37b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f34a8100e50 tx=0x7f34a0009310 comp rx=0 tx=0).stop 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a8101990 msgr2=0x7f34a819c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a8101990 0x7f34a819c380 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f34a400b6c0 tx=0x7f34a4007730 comp rx=0 tx=0).stop 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 shutdown_connections 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34940a12f0 0x7f34940a37b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34a80ffe00 0x7f34a8194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34a8101990 0x7f34a819c380 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 --2- 172.21.15.64:0/3794390131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a8100ff0 0x7f34a8195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 >> 172.21.15.64:0/3794390131 conn(0x7f34a80fb390 msgr2=0x7f34a80fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 shutdown_connections 2024-05-08T22:05:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:01.463+0000 7f34b040f700 1 -- 172.21.15.64:0/3794390131 wait complete. 2024-05-08T22:05:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 610.0 s, 1292.8 tps, lat 77.143 ms stddev 126.600 2024-05-08T22:05:03.796 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:03 smithi160 ceph-mon[46962]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.89k op/s 2024-05-08T22:05:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:03 smithi186 ceph-mon[46619]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.89k op/s 2024-05-08T22:05:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:03 smithi064 ceph-mon[42500]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.89k op/s 2024-05-08T22:05:04.795 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:04 smithi160 ceph-mon[46962]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2024-05-08T22:05:04.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:04 smithi186 ceph-mon[46619]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2024-05-08T22:05:04.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:04 smithi064 ceph-mon[42500]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2024-05-08T22:05:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 615.0 s, 1626.4 tps, lat 60.669 ms stddev 76.791 2024-05-08T22:05:06.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:06 smithi186 ceph-mon[46619]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.94k op/s 2024-05-08T22:05:06.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:06 smithi064 ceph-mon[42500]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.94k op/s 2024-05-08T22:05:07.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:06 smithi160 ceph-mon[46962]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.94k op/s 2024-05-08T22:05:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:08 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:05:08.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:08 smithi186 ceph-mon[46619]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 MiB/s wr, 1.82k op/s 2024-05-08T22:05:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:08 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:05:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:08 smithi064 ceph-mon[42500]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 MiB/s wr, 1.82k op/s 2024-05-08T22:05:09.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:08 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:05:09.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:08 smithi160 ceph-mon[46962]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 MiB/s wr, 1.82k op/s 2024-05-08T22:05:10.898 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:10 smithi186 ceph-mon[46619]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 2.01k op/s 2024-05-08T22:05:10.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:10 smithi064 ceph-mon[42500]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 2.01k op/s 2024-05-08T22:05:11.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:10 smithi160 ceph-mon[46962]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 2.01k op/s 2024-05-08T22:05:11.481 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 620.0 s, 1667.1 tps, lat 60.376 ms stddev 70.709 2024-05-08T22:05:12.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:12 smithi186 ceph-mon[46619]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-05-08T22:05:12.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:12 smithi064 ceph-mon[42500]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-05-08T22:05:13.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:12 smithi160 ceph-mon[46962]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-05-08T22:05:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 -- 172.21.15.64:0/776037470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c102a30 msgr2=0x7f918c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 --2- 172.21.15.64:0/776037470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c102a30 0x7f918c109350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f918400b0a0 tx=0x7f918400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 -- 172.21.15.64:0/776037470 shutdown_connections 2024-05-08T22:05:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 --2- 172.21.15.64:0/776037470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c102a30 0x7f918c109350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 --2- 172.21.15.64:0/776037470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c100f20 0x7f918c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 --2- 172.21.15.64:0/776037470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102070 0x7f918c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 -- 172.21.15.64:0/776037470 >> 172.21.15.64:0/776037470 conn(0x7f918c0fc490 msgr2=0x7f918c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 -- 172.21.15.64:0/776037470 shutdown_connections 2024-05-08T22:05:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.636+0000 7f9191450700 1 -- 172.21.15.64:0/776037470 wait complete. 2024-05-08T22:05:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 Processor -- start 2024-05-08T22:05:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 -- start start 2024-05-08T22:05:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c100f20 0x7f918c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c102070 0x7f918c19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102a30 0x7f918c1a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a4fb0 con 0x7f918c102070 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a50f0 con 0x7f918c102a30 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.637+0000 7f9191450700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a5280 con 0x7f918c100f20 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c102070 0x7f918c19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c102070 0x7f918c19d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37696/0 (socket says 172.21.15.64:37696) 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918b7fe700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102a30 0x7f918c1a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918a7fc700 1 -- 172.21.15.64:0/2621408812 learned_addr learned my addr 172.21.15.64:0/2621408812 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c100f20 0x7f918c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 -- 172.21.15.64:0/2621408812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102a30 msgr2=0x7f918c1a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102a30 0x7f918c1a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 -- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c102070 msgr2=0x7f918c19d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c102070 0x7f918c19d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 -- 172.21.15.64:0/2621408812 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184009d00 con 0x7f918c100f20 2024-05-08T22:05:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918b7fe700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102a30 0x7f918c1a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f918affd700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c100f20 0x7f918c19d480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f917c00b450 tx=0x7f917c00b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f9173fff700 1 -- 172.21.15.64:0/2621408812 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011830 con 0x7f918c100f20 2024-05-08T22:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f9173fff700 1 -- 172.21.15.64:0/2621408812 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917c00be60 con 0x7f918c100f20 2024-05-08T22:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c1a5560 con 0x7f918c100f20 2024-05-08T22:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f9173fff700 1 -- 172.21.15.64:0/2621408812 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c010cb0 con 0x7f918c100f20 2024-05-08T22:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.638+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c1a5b40 con 0x7f918c100f20 2024-05-08T22:05:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.641+0000 7f9173fff700 1 -- 172.21.15.64:0/2621408812 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f917c011990 con 0x7f918c100f20 2024-05-08T22:05:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.642+0000 7f9173fff700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f91740a11d0 0x7f91740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.642+0000 7f9173fff700 1 -- 172.21.15.64:0/2621408812 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f917c0cc130 con 0x7f918c100f20 2024-05-08T22:05:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.642+0000 7f918a7fc700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f91740a11d0 0x7f91740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.642+0000 7f919d60ab80 1 Processor -- start 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 -- start start 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f91740a11d0 0x7f91740a3690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9180009ad0 tx=0x7f9180009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a36a80 0x55be58a36ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a37cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a382d0 0x55be58a38750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5898e0f0 con 0x55be58a373e0 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5898bab0 con 0x55be58a382d0 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f919d60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be589873a0 con 0x55be58a36a80 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a37cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a37cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37710/0 (socket says 172.21.15.64:37710) 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 -- 172.21.15.64:0/3201562555 learned_addr learned my addr 172.21.15.64:0/3201562555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918affd700 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a36a80 0x55be58a36ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918b7fe700 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a382d0 0x55be58a38750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 -- 172.21.15.64:0/3201562555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a36a80 msgr2=0x55be58a36ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a36a80 0x55be58a36ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 -- 172.21.15.64:0/3201562555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a382d0 msgr2=0x55be58a38750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a382d0 0x55be58a38750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.643+0000 7f918a7fc700 1 -- 172.21.15.64:0/3201562555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be58808350 con 0x55be58a373e0 2024-05-08T22:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.644+0000 7f918a7fc700 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a37cf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f918000eac0 tx=0x7f9180011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147151fde5b7040b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.645+0000 7f91717fa700 1 -- 172.21.15.64:0/3201562555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180015480 con 0x55be58a373e0 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.645+0000 7f91717fa700 1 -- 172.21.15.64:0/3201562555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9180015aa0 con 0x55be58a373e0 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.645+0000 7f91717fa700 1 -- 172.21.15.64:0/3201562555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180013ab0 con 0x55be58a373e0 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 -- 172.21.15.64:0/3201562555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 msgr2=0x55be58a37cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a37cf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f918000eac0 tx=0x7f9180011000 comp rx=0 tx=0).stop 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 -- 172.21.15.64:0/3201562555 shutdown_connections 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a382d0 0x55be58a38750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a36a80 0x55be58a36ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 --2- 172.21.15.64:0/3201562555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a37cf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 -- 172.21.15.64:0/3201562555 >> 172.21.15.64:0/3201562555 conn(0x55be589951b0 msgr2=0x55be58a33440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 -- 172.21.15.64:0/3201562555 shutdown_connections 2024-05-08T22:05:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.646+0000 7f919d60ab80 1 -- 172.21.15.64:0/3201562555 wait complete. 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.647+0000 7f919d60ab80 1 -- start start 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.647+0000 7f919d60ab80 1 Processor -- start 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a40e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a47e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58923500 con 0x55be58a373e0 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be589231a0 con 0x55be58a36a80 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5880a640 con 0x55be58a382d0 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f919d60ab80 10 client.0 _ll_register_callbacks cb 0x55be5892cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a40e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:44990/0 (socket says 172.21.15.64:44990) 2024-05-08T22:05:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 -- 172.21.15.64:0/831329887 learned_addr learned my addr 172.21.15.64:0/831329887 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918b7fe700 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a47e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 -- 172.21.15.64:0/831329887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 msgr2=0x55be58a47e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a47e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 -- 172.21.15.64:0/831329887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 msgr2=0x55be58a40e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a40e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 -- 172.21.15.64:0/831329887 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918000e7a0 con 0x55be58a36a80 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f918affd700 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a408f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f917c095a90 tx=0x7f917c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b5d714d3a14996 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.648+0000 7f916eff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f916eff5700 1 -- 172.21.15.64:0/831329887 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0b1070 con 0x55be58a36a80 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f916eff5700 1 -- 172.21.15.64:0/831329887 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f917c0b4060 con 0x55be58a36a80 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be588e0210 con 0x55be58a36a80 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f916eff5700 1 -- 172.21.15.64:0/831329887 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f917c09d1b0 con 0x55be58a36a80 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f919d60ab80 10 client.25754 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55be587b4b00 con 0x55be58a36a80 2024-05-08T22:05:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.649+0000 7f916eff5700 1 -- 172.21.15.64:0/831329887 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be588e0210 con 0x55be58a36a80 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f916eff5700 1 -- 172.21.15.64:0/831329887 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f917c0997f0 con 0x55be58a36a80 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f919d60ab80 10 client.25754 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f919d60ab80 10 client.25754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f919d60ab80 20 client.25754 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f919d60ab80 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58a5d6e0 0x55be58a5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f919d60ab80 4 client.25754 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be58a067b0 con 0x55be58a5d6e0 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f918a7fc700 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58a5d6e0 0x55be58a5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f916eff5700 10 client.25754 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.650+0000 7f918a7fc700 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58a5d6e0 0x55be58a5fb00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9da8467eaef4013 server_cookie=17be1a56a46e7c28 in_seq=0 out_seq=0 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.652+0000 7f916eff5700 1 -- 172.21.15.64:0/831329887 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be58a067b0 con 0x55be58a5d6e0 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.652+0000 7f916eff5700 10 client.25754 handle_command_reply: tid=0 2024-05-08T22:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.652+0000 7f919d60ab80 1 client.25754 shutdown 2024-05-08T22:05:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 msgr2=0x55be58a408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a408f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f917c095a90 tx=0x7f917c004750 comp rx=0 tx=0).stop 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 shutdown_connections 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a408f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58a5d6e0 0x55be58a5fb00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 --2- 172.21.15.64:0/831329887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a40e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.653+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 >> 172.21.15.64:0/831329887 conn(0x55be589951b0 msgr2=0x55be58a31860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.658+0000 7f916eff5700 0 client.25754 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.658+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 shutdown_connections 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.658+0000 7f919d60ab80 1 -- 172.21.15.64:0/831329887 wait complete. 2024-05-08T22:05:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.658+0000 7f919d60ab80 20 client.25754 trim_cache size 0 max 16384 2024-05-08T22:05:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.681+0000 7f919d60ab80 1 Processor -- start 2024-05-08T22:05:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 -- start start 2024-05-08T22:05:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a3fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58a2e9b0 con 0x55be58a373e0 2024-05-08T22:05:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58a5ac10 con 0x55be58a36a80 2024-05-08T22:05:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f919d60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58a5a000 con 0x55be58a382d0 2024-05-08T22:05:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.693+0000 7f918affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.694+0000 7f918affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45002/0 (socket says 172.21.15.64:45002) 2024-05-08T22:05:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.694+0000 7f918affd700 1 -- 172.21.15.64:0/1252763870 learned_addr learned my addr 172.21.15.64:0/1252763870 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.695+0000 7f918affd700 1 -- 172.21.15.64:0/1252763870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 msgr2=0x55be58a3fe00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:05:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.695+0000 7f918affd700 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a3fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.695+0000 7f918affd700 1 -- 172.21.15.64:0/1252763870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 msgr2=0x55be58a47e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.695+0000 7f918affd700 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.695+0000 7f918affd700 1 -- 172.21.15.64:0/1252763870 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be5894d500 con 0x55be58a36a80 2024-05-08T22:05:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.715+0000 7f918affd700 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a386b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f917c0978e0 tx=0x7f917c010690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cabdc253354e0571 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.727+0000 7f9171ffb700 1 -- 172.21.15.64:0/1252763870 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0b1070 con 0x55be58a36a80 2024-05-08T22:05:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.727+0000 7f9171ffb700 1 -- 172.21.15.64:0/1252763870 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917c0b3020 con 0x55be58a36a80 2024-05-08T22:05:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.727+0000 7f9171ffb700 1 -- 172.21.15.64:0/1252763870 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c099600 con 0x55be58a36a80 2024-05-08T22:05:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.728+0000 7f919d60ab80 1 -- 172.21.15.64:0/1252763870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 msgr2=0x55be58a386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.728+0000 7f919d60ab80 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a386b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f917c0978e0 tx=0x7f917c010690 comp rx=0 tx=0).stop 2024-05-08T22:05:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.732+0000 7f919d60ab80 1 -- 172.21.15.64:0/1252763870 shutdown_connections 2024-05-08T22:05:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.732+0000 7f919d60ab80 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a36a80 0x55be58a386b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.732+0000 7f919d60ab80 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58a3fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.732+0000 7f919d60ab80 1 --2- 172.21.15.64:0/1252763870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a373e0 0x55be58a47e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.732+0000 7f919d60ab80 1 -- 172.21.15.64:0/1252763870 >> 172.21.15.64:0/1252763870 conn(0x55be589951b0 msgr2=0x55be58a3e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.734+0000 7f919d60ab80 1 -- 172.21.15.64:0/1252763870 shutdown_connections 2024-05-08T22:05:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.734+0000 7f919d60ab80 1 -- 172.21.15.64:0/1252763870 wait complete. 2024-05-08T22:05:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.734+0000 7f919d60ab80 1 -- start start 2024-05-08T22:05:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.734+0000 7f919d60ab80 1 Processor -- start 2024-05-08T22:05:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.734+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a36a80 0x55be58a430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.734+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 0x55be58a435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f919d60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58ac2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f919d60ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58a48020 con 0x55be58a36a80 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f919d60ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58a48190 con 0x55be58a373e0 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f919d60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be58ac3110 con 0x55be58a382d0 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f919d60ab80 10 client.0 _ll_register_callbacks cb 0x55be5892cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 0x55be58a435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f918a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 0x55be58a435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:45004/0 (socket says 172.21.15.64:45004) 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f918a7fc700 1 -- 172.21.15.64:0/2821768129 learned_addr learned my addr 172.21.15.64:0/2821768129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.735+0000 7f918affd700 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a36a80 0x55be58a430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.736+0000 7f918a7fc700 1 -- 172.21.15.64:0/2821768129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 msgr2=0x55be58ac2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.736+0000 7f918a7fc700 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58ac2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.736+0000 7f918a7fc700 1 -- 172.21.15.64:0/2821768129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a36a80 msgr2=0x55be58a430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.736+0000 7f918a7fc700 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a36a80 0x55be58a430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.736+0000 7f918a7fc700 1 -- 172.21.15.64:0/2821768129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be587b4b00 con 0x55be58a373e0 2024-05-08T22:05:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.741+0000 7f918a7fc700 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 0x55be58a435e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9180009790 tx=0x7f9180015480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f420773b2e8a060 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.741+0000 7f916eff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:05:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.747+0000 7f916eff5700 1 -- 172.21.15.64:0/2821768129 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180038070 con 0x55be58a373e0 2024-05-08T22:05:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.747+0000 7f916eff5700 1 -- 172.21.15.64:0/2821768129 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9180015830 con 0x55be58a373e0 2024-05-08T22:05:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.747+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9180015aa0 con 0x55be58a373e0 2024-05-08T22:05:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.750+0000 7f916eff5700 1 -- 172.21.15.64:0/2821768129 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918002e040 con 0x55be58a373e0 2024-05-08T22:05:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.750+0000 7f919d60ab80 10 client.25760 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.750+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55be5880cc70 con 0x55be58a373e0 2024-05-08T22:05:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.751+0000 7f916eff5700 1 -- 172.21.15.64:0/2821768129 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9180015aa0 con 0x55be58a373e0 2024-05-08T22:05:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.751+0000 7f916eff5700 1 -- 172.21.15.64:0/2821768129 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f918003b020 con 0x55be58a373e0 2024-05-08T22:05:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.752+0000 7f919d60ab80 10 client.25760 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.752+0000 7f919d60ab80 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.752+0000 7f919d60ab80 20 client.25760 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.752+0000 7f919d60ab80 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58ac49e0 0x55be58ac6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.752+0000 7f919d60ab80 4 client.25760 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:05:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.752+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be5897fe60 con 0x55be58ac49e0 2024-05-08T22:05:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.753+0000 7f918affd700 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58ac49e0 0x55be58ac6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.754+0000 7f918affd700 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58ac49e0 0x55be58ac6e00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df861b66bccd168b server_cookie=d05443bfd70a3baf in_seq=0 out_seq=0 2024-05-08T22:05:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.754+0000 7f916eff5700 10 client.25760 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f916eff5700 1 -- 172.21.15.64:0/2821768129 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be5897fe60 con 0x55be58ac49e0 2024-05-08T22:05:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f916eff5700 10 client.25760 handle_command_reply: tid=0 2024-05-08T22:05:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 client.25760 shutdown 2024-05-08T22:05:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 msgr2=0x55be58a435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 0x55be58a435e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9180009790 tx=0x7f9180015480 comp rx=0 tx=0).stop 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 shutdown_connections 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55be58ac49e0 0x55be58ac6e00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be58a382d0 0x55be58ac2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be58a36a80 0x55be58a430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 --2- 172.21.15.64:0/2821768129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be58a373e0 0x55be58a435e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 >> 172.21.15.64:0/2821768129 conn(0x55be589951b0 msgr2=0x55be58a5f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f916eff5700 0 client.25760 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.782+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 shutdown_connections 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.783+0000 7f919d60ab80 1 -- 172.21.15.64:0/2821768129 wait complete. 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.783+0000 7f919d60ab80 20 client.25760 trim_cache size 0 max 16384 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:05:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:05:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:05:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f91740a11d0 msgr2=0x7f91740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f91740a11d0 0x7f91740a3690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9180009ad0 tx=0x7f9180009210 comp rx=0 tx=0).stop 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c100f20 msgr2=0x7f918c19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c100f20 0x7f918c19d480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f917c00b450 tx=0x7f917c00b760 comp rx=0 tx=0).stop 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 shutdown_connections 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f91740a11d0 0x7f91740a3690 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102a30 0x7f918c1a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f918c100f20 0x7f918c19d480 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f917c00b450 tx=0x7f917c00b760 comp rx=0 tx=0).stop 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 --2- 172.21.15.64:0/2621408812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f918c102070 0x7f918c19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.784+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 >> 172.21.15.64:0/2621408812 conn(0x7f918c0fc490 msgr2=0x7f918c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.785+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 shutdown_connections 2024-05-08T22:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:14.785+0000 7f9191450700 1 -- 172.21.15.64:0/2621408812 wait complete. 2024-05-08T22:05:14.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:14 smithi186 ceph-mon[46619]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 702 op/s 2024-05-08T22:05:14.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:14 smithi064 ceph-mon[42500]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 702 op/s 2024-05-08T22:05:15.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:14 smithi160 ceph-mon[46962]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 702 op/s 2024-05-08T22:05:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 625.0 s, 1968.7 tps, lat 51.258 ms stddev 61.948 2024-05-08T22:05:16.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:16 smithi186 ceph-mon[46619]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 691 op/s 2024-05-08T22:05:16.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:16 smithi064 ceph-mon[42500]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 691 op/s 2024-05-08T22:05:17.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:16 smithi160 ceph-mon[46962]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 691 op/s 2024-05-08T22:05:18.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:18 smithi186 ceph-mon[46619]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.9 MiB/s wr, 589 op/s 2024-05-08T22:05:18.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:18 smithi064 ceph-mon[42500]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.9 MiB/s wr, 589 op/s 2024-05-08T22:05:19.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:18 smithi160 ceph-mon[46962]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.9 MiB/s wr, 589 op/s 2024-05-08T22:05:20.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:20 smithi186 ceph-mon[46619]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-08T22:05:20.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:20 smithi064 ceph-mon[42500]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-08T22:05:21.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:20 smithi160 ceph-mon[46962]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-08T22:05:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 630.0 s, 1830.4 tps, lat 54.792 ms stddev 65.117 2024-05-08T22:05:22.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:22 smithi186 ceph-mon[46619]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-05-08T22:05:22.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:22 smithi064 ceph-mon[42500]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-05-08T22:05:23.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:22 smithi160 ceph-mon[46962]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-05-08T22:05:24.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:24 smithi186 ceph-mon[46619]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 650 op/s 2024-05-08T22:05:24.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:24 smithi064 ceph-mon[42500]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 650 op/s 2024-05-08T22:05:25.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:24 smithi160 ceph-mon[46962]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 650 op/s 2024-05-08T22:05:26.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 635.0 s, 1919.6 tps, lat 51.824 ms stddev 58.394 2024-05-08T22:05:26.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:26 smithi064 ceph-mon[42500]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 704 op/s 2024-05-08T22:05:27.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:26 smithi160 ceph-mon[46962]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 704 op/s 2024-05-08T22:05:27.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:26 smithi186 ceph-mon[46619]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 704 op/s 2024-05-08T22:05:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 -- 172.21.15.64:0/3453063440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 msgr2=0x7fca1c100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 --2- 172.21.15.64:0/3453063440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 0x7fca1c100260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009990 tx=0x7fca0c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 -- 172.21.15.64:0/3453063440 shutdown_connections 2024-05-08T22:05:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 --2- 172.21.15.64:0/3453063440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca1c101950 0x7fca1c108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 --2- 172.21.15.64:0/3453063440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 0x7fca1c100260 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 --2- 172.21.15.64:0/3453063440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c100f90 0x7fca1c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 -- 172.21.15.64:0/3453063440 >> 172.21.15.64:0/3453063440 conn(0x7fca1c0fb3b0 msgr2=0x7fca1c0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 -- 172.21.15.64:0/3453063440 shutdown_connections 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.948+0000 7fca21ee0700 1 -- 172.21.15.64:0/3453063440 wait complete. 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 Processor -- start 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 -- start start 2024-05-08T22:05:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 0x7fca1c06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca1c100f90 0x7fca1c06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 0x7fca1c072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0728d0 con 0x7fca1c0ffe40 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c072a40 con 0x7fca1c100f90 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.949+0000 7fca21ee0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c072c00 con 0x7fca1c101950 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 0x7fca1c072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 0x7fca1c072300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:44218/0 (socket says 172.21.15.64:44218) 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 -- 172.21.15.64:0/2354172956 learned_addr learned my addr 172.21.15.64:0/2354172956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 -- 172.21.15.64:0/2354172956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca1c100f90 msgr2=0x7fca1c06b2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca1c100f90 0x7fca1c06b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 -- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 msgr2=0x7fca1c06ada0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 0x7fca1c06ada0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 -- 172.21.15.64:0/2354172956 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c0095f0 con 0x7fca1c101950 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.950+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 0x7fca1c072300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fca1400ef30 tx=0x7fca1400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.951+0000 7fca18ff9700 1 -- 172.21.15.64:0/2354172956 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14018070 con 0x7fca1c101950 2024-05-08T22:05:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.951+0000 7fca18ff9700 1 -- 172.21.15.64:0/2354172956 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1400f040 con 0x7fca1c101950 2024-05-08T22:05:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.951+0000 7fca18ff9700 1 -- 172.21.15.64:0/2354172956 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14003dc0 con 0x7fca1c101950 2024-05-08T22:05:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.951+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c072e20 con 0x7fca1c101950 2024-05-08T22:05:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.951+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a6ba0 con 0x7fca1c101950 2024-05-08T22:05:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.954+0000 7fca18ff9700 1 -- 172.21.15.64:0/2354172956 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fca1400caa0 con 0x7fca1c101950 2024-05-08T22:05:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.954+0000 7fca18ff9700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fca040a1270 0x7fca040a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca18ff9700 1 -- 172.21.15.64:0/2354172956 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca140cc4b0 con 0x7fca1c101950 2024-05-08T22:05:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fca040a1270 0x7fca040a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 Processor -- start 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 -- start start 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafdebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafdf9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafdfc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafd430f0 con 0x564bafdec130 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafd40ab0 con 0x564bafdeb7d0 2024-05-08T22:05:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafd3c3a0 con 0x564bafdfa0a0 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca1affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafdf9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca1affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafdf9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46660/0 (socket says 172.21.15.64:46660) 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.955+0000 7fca1affd700 1 -- 172.21.15.64:0/1256120852 learned_addr learned my addr 172.21.15.64:0/1256120852 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafdebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1bfff700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafdfc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fca040a1270 0x7fca040a3730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca0c006010 tx=0x7fca0c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 -- 172.21.15.64:0/1256120852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 msgr2=0x564bafdfc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafdfc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 -- 172.21.15.64:0/1256120852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 msgr2=0x564bafdf9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafdf9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 -- 172.21.15.64:0/1256120852 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bafbbd350 con 0x564bafdeb7d0 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1bfff700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafdfc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1affd700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafdf9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafdebbf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b620 tx=0x7fca0c027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94627e0d4b22337 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca01ffb700 1 -- 172.21.15.64:0/1256120852 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00f6e0 con 0x564bafdeb7d0 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca01ffb700 1 -- 172.21.15.64:0/1256120852 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0c00fd00 con 0x564bafdeb7d0 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.956+0000 7fca01ffb700 1 -- 172.21.15.64:0/1256120852 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c027d30 con 0x564bafdeb7d0 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.957+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1256120852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 msgr2=0x564bafdebbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.957+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafdebbf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b620 tx=0x7fca0c027000 comp rx=0 tx=0).stop 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.958+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1256120852 shutdown_connections 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.958+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafdebbf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.958+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafdfc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.958+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1256120852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafdf9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.958+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1256120852 >> 172.21.15.64:0/1256120852 conn(0x564bafd4a1b0 msgr2=0x564bafde8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.959+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1256120852 shutdown_connections 2024-05-08T22:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.959+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1256120852 wait complete. 2024-05-08T22:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.959+0000 7fca2e09ab80 1 -- start start 2024-05-08T22:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 Processor -- start 2024-05-08T22:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdfdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafdfe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe05490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafcd8500 con 0x564bafdfa0a0 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafcd81a0 con 0x564bafdec130 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafbbf640 con 0x564bafdeb7d0 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca2e09ab80 10 client.0 _ll_register_callbacks cb 0x564bafce1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca1b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdfdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe05490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca1b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdfdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:44236/0 (socket says 172.21.15.64:44236) 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.960+0000 7fca1b7fe700 1 -- 172.21.15.64:0/3248896095 learned_addr learned my addr 172.21.15.64:0/3248896095 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca1bfff700 1 -- 172.21.15.64:0/3248896095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 msgr2=0x564bafdfdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdfdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca1bfff700 1 -- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 msgr2=0x564bafdfe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafdfe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca1bfff700 1 -- 172.21.15.64:0/3248896095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a6ba0 con 0x564bafdfa0a0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe05490 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca140c9f60 tx=0x7fca1401b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84f9590808cf784 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fc9faffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fc9faffd700 1 -- 172.21.15.64:0/3248896095 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140b7ea0 con 0x564bafdfa0a0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fc9faffd700 1 -- 172.21.15.64:0/3248896095 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca140b2220 con 0x564bafdfa0a0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.961+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bafc95210 con 0x564bafdfa0a0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fc9faffd700 1 -- 172.21.15.64:0/3248896095 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca1409d770 con 0x564bafdfa0a0 2024-05-08T22:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 10 client.15981 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fca1c108ba0 con 0x564bafdfa0a0 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fc9faffd700 1 -- 172.21.15.64:0/3248896095 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bafc95210 con 0x564bafdfa0a0 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fc9faffd700 1 -- 172.21.15.64:0/3248896095 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fca14098e30 con 0x564bafdfa0a0 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 10 client.15981 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 20 client.15981 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe09d70 0x564bafe0c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 4 client.15981 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.962+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bafdbb7b0 con 0x564bafe09d70 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.963+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe09d70 0x564bafe0c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.964+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe09d70 0x564bafe0c190 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05324b8c3d6aea8 server_cookie=2432f29c08e09d86 in_seq=0 out_seq=0 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.964+0000 7fc9faffd700 10 client.15981 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.964+0000 7fc9faffd700 1 -- 172.21.15.64:0/3248896095 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564bafdbb7b0 con 0x564bafe09d70 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.964+0000 7fc9faffd700 10 client.15981 handle_command_reply: tid=0 2024-05-08T22:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.964+0000 7fca2e09ab80 1 client.15981 shutdown 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 msgr2=0x564bafe05490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe05490 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca140c9f60 tx=0x7fca1401b600 comp rx=0 tx=0).stop 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 shutdown_connections 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe09d70 0x564bafe0c190 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe05490 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdfdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3248896095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafdfe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 >> 172.21.15.64:0/3248896095 conn(0x564bafd4a1b0 msgr2=0x564bafde6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fc9faffd700 0 client.15981 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 shutdown_connections 2024-05-08T22:05:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3248896095 wait complete. 2024-05-08T22:05:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.965+0000 7fca2e09ab80 20 client.15981 trim_cache size 0 max 16384 2024-05-08T22:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.984+0000 7fca2e09ab80 1 Processor -- start 2024-05-08T22:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.984+0000 7fca2e09ab80 1 -- start start 2024-05-08T22:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdf97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafe05450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe10af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafde1330 con 0x564bafdfa0a0 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafe07220 con 0x564bafdec130 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafdfee10 con 0x564bafdeb7d0 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe10af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe10af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46668/0 (socket says 172.21.15.64:46668) 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafe05450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 -- 172.21.15.64:0/1327658138 learned_addr learned my addr 172.21.15.64:0/1327658138 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdf97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 -- 172.21.15.64:0/1327658138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 msgr2=0x564bafdf97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdf97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 -- 172.21.15.64:0/1327658138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 msgr2=0x564bafe05450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafe05450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 -- 172.21.15.64:0/1327658138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bafb69b00 con 0x564bafdfa0a0 2024-05-08T22:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1affd700 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafe05450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.985+0000 7fca1bfff700 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe10af0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c850 tx=0x7fca14095db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d5305df37db78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca027fc700 1 -- 172.21.15.64:0/1327658138 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140b7ea0 con 0x564bafdfa0a0 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca027fc700 1 -- 172.21.15.64:0/1327658138 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca140b2220 con 0x564bafdfa0a0 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca027fc700 1 -- 172.21.15.64:0/1327658138 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14097e00 con 0x564bafdfa0a0 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1327658138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 msgr2=0x564bafe10af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe10af0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c850 tx=0x7fca14095db0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1327658138 shutdown_connections 2024-05-08T22:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdfa0a0 0x564bafe10af0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdeb7d0 0x564bafdf97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/1327658138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdec130 0x564bafe05450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1327658138 >> 172.21.15.64:0/1327658138 conn(0x564bafd4a1b0 msgr2=0x564bafdfcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1327658138 shutdown_connections 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.986+0000 7fca2e09ab80 1 -- 172.21.15.64:0/1327658138 wait complete. 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 -- start start 2024-05-08T22:05:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 Processor -- start 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafe02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafe02ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafe03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafe04150 con 0x564bafdec130 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafe042c0 con 0x564bafdeb7d0 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bafe04480 con 0x564bafdfa0a0 2024-05-08T22:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca2e09ab80 10 client.0 _ll_register_callbacks cb 0x564bafce1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca1affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafe02ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca1affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafe02ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46674/0 (socket says 172.21.15.64:46674) 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca1affd700 1 -- 172.21.15.64:0/3328327734 learned_addr learned my addr 172.21.15.64:0/3328327734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafe02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.987+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafe03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1affd700 1 -- 172.21.15.64:0/3328327734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 msgr2=0x564bafe03c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1affd700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafe03c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1affd700 1 -- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 msgr2=0x564bafe02760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1affd700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafe02760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1affd700 1 -- 172.21.15.64:0/3328327734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca10007860 con 0x564bafdec130 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafe03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafe02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca1affd700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafe02ca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fca10004790 tx=0x7fca10011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bbc74d15476df5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fc9faffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:05:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fc9faffd700 1 -- 172.21.15.64:0/3328327734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10016070 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fc9faffd700 1 -- 172.21.15.64:0/3328327734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca10004830 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca0c00fd00 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fc9faffd700 1 -- 172.21.15.64:0/3328327734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca10021030 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca2e09ab80 10 client.15987 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fca1c108ba0 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.988+0000 7fc9faffd700 1 -- 172.21.15.64:0/3328327734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0c00fd00 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fc9faffd700 1 -- 172.21.15.64:0/3328327734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fca1001b870 con 0x564bafdec130 2024-05-08T22:05:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca2e09ab80 10 client.15987 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca2e09ab80 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca2e09ab80 20 client.15987 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe607c0 0x564bafe62be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca2e09ab80 4 client.15987 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564bafd34e60 con 0x564bafe607c0 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.989+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe607c0 0x564bafe62be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.991+0000 7fc9faffd700 10 client.15987 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:27.991+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe607c0 0x564bafe62be0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b5f27297a8de900 server_cookie=13c64ac32f93bb4d in_seq=0 out_seq=0 2024-05-08T22:05:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.087+0000 7fc9faffd700 1 -- 172.21.15.64:0/3328327734 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564bafd34e60 con 0x564bafe607c0 2024-05-08T22:05:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.087+0000 7fc9faffd700 10 client.15987 handle_command_reply: tid=0 2024-05-08T22:05:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.087+0000 7fca2e09ab80 1 client.15987 shutdown 2024-05-08T22:05:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.087+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 msgr2=0x564bafe02ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.087+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafe02ca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fca10004790 tx=0x7fca10011720 comp rx=0 tx=0).stop 2024-05-08T22:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 shutdown_connections 2024-05-08T22:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564bafdeb7d0 0x564bafe02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x564bafe607c0 0x564bafe62be0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bafdfa0a0 0x564bafe03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 --2- 172.21.15.64:0/3328327734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bafdec130 0x564bafe02ca0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 >> 172.21.15.64:0/3328327734 conn(0x564bafd4a1b0 msgr2=0x564bafdf9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fc9faffd700 0 client.15987 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 shutdown_connections 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 1 -- 172.21.15.64:0/3328327734 wait complete. 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.088+0000 7fca2e09ab80 20 client.15987 trim_cache size 0 max 16384 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fca040a1270 msgr2=0x7fca040a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fca040a1270 0x7fca040a3730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca0c006010 tx=0x7fca0c005be0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 msgr2=0x7fca1c072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 0x7fca1c072300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fca1400ef30 tx=0x7fca1400c450 comp rx=0 tx=0).stop 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 shutdown_connections 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fca040a1270 0x7fca040a3730 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca1c101950 0x7fca1c072300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c0ffe40 0x7fca1c06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 --2- 172.21.15.64:0/2354172956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca1c100f90 0x7fca1c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 >> 172.21.15.64:0/2354172956 conn(0x7fca1c0fb3b0 msgr2=0x7fca1c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 shutdown_connections 2024-05-08T22:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:28.089+0000 7fca21ee0700 1 -- 172.21.15.64:0/2354172956 wait complete. 2024-05-08T22:05:28.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:28 smithi186 ceph-mon[46619]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 MiB/s wr, 598 op/s 2024-05-08T22:05:28.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:28 smithi064 ceph-mon[42500]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 MiB/s wr, 598 op/s 2024-05-08T22:05:29.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:28 smithi160 ceph-mon[46962]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 MiB/s wr, 598 op/s 2024-05-08T22:05:30.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:30 smithi186 ceph-mon[46619]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 733 op/s 2024-05-08T22:05:30.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:30 smithi064 ceph-mon[42500]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 733 op/s 2024-05-08T22:05:31.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:30 smithi160 ceph-mon[46962]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 733 op/s 2024-05-08T22:05:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 640.0 s, 2086.0 tps, lat 47.794 ms stddev 59.205 2024-05-08T22:05:32.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:32 smithi186 ceph-mon[46619]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 699 op/s 2024-05-08T22:05:32.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:32 smithi064 ceph-mon[42500]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 699 op/s 2024-05-08T22:05:33.051 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:32 smithi160 ceph-mon[46962]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 699 op/s 2024-05-08T22:05:34.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:34 smithi186 ceph-mon[46619]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.7 MiB/s wr, 686 op/s 2024-05-08T22:05:34.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:34 smithi064 ceph-mon[42500]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.7 MiB/s wr, 686 op/s 2024-05-08T22:05:35.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:34 smithi160 ceph-mon[46962]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.7 MiB/s wr, 686 op/s 2024-05-08T22:05:35.904 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:35 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:35.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:35 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:35 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 645.0 s, 1353.4 tps, lat 73.680 ms stddev 78.689 2024-05-08T22:05:36.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:36 smithi186 ceph-mon[46619]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 753 op/s 2024-05-08T22:05:36.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:36 smithi064 ceph-mon[42500]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 753 op/s 2024-05-08T22:05:37.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:36 smithi160 ceph-mon[46962]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 753 op/s 2024-05-08T22:05:37.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:37 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:37 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:37 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:38 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:38.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:38 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:38.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:38 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:38 smithi186 ceph-mon[46619]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.0 MiB/s wr, 605 op/s 2024-05-08T22:05:38.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:38 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:38.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:38 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:38.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:38 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:38.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:38 smithi064 ceph-mon[42500]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.0 MiB/s wr, 605 op/s 2024-05-08T22:05:39.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:38 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:39.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:38 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:39.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:38 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:05:39.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:38 smithi160 ceph-mon[46962]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.0 MiB/s wr, 605 op/s 2024-05-08T22:05:40.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:40 smithi186 ceph-mon[46619]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.1 MiB/s wr, 717 op/s 2024-05-08T22:05:40.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:40 smithi064 ceph-mon[42500]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.1 MiB/s wr, 717 op/s 2024-05-08T22:05:41.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:40 smithi160 ceph-mon[46962]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.1 MiB/s wr, 717 op/s 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 -- 172.21.15.64:0/1056423647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 msgr2=0x7ff274108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 --2- 172.21.15.64:0/1056423647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 0x7ff274108240 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b0d0 tx=0x7ff26c00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 -- 172.21.15.64:0/1056423647 shutdown_connections 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 --2- 172.21.15.64:0/1056423647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 0x7ff274108240 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 --2- 172.21.15.64:0/1056423647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2740ffe00 0x7ff274100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 --2- 172.21.15.64:0/1056423647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274100ff0 0x7ff274101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 -- 172.21.15.64:0/1056423647 >> 172.21.15.64:0/1056423647 conn(0x7ff2740fb390 msgr2=0x7ff2740fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 -- 172.21.15.64:0/1056423647 shutdown_connections 2024-05-08T22:05:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.255+0000 7ff279751700 1 -- 172.21.15.64:0/1056423647 wait complete. 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 Processor -- start 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 -- start start 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 0x7ff274068da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff274100ff0 0x7ff27419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 0x7ff2741a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741a0c30 con 0x7ff274101990 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741a0da0 con 0x7ff2740ffe00 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff279751700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741a0f60 con 0x7ff274100ff0 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff272ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 0x7ff274068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff272ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 0x7ff274068da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40196/0 (socket says 172.21.15.64:40196) 2024-05-08T22:05:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff272ffd700 1 -- 172.21.15.64:0/1930740446 learned_addr learned my addr 172.21.15.64:0/1930740446 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.256+0000 7ff2727fc700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff274100ff0 0x7ff27419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.257+0000 7ff2737fe700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 0x7ff2741a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.257+0000 7ff272ffd700 1 -- 172.21.15.64:0/1930740446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff274100ff0 msgr2=0x7ff27419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.257+0000 7ff272ffd700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff274100ff0 0x7ff27419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.257+0000 7ff272ffd700 1 -- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 msgr2=0x7ff2741a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.257+0000 7ff272ffd700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 0x7ff2741a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.257+0000 7ff272ffd700 1 -- 172.21.15.64:0/1930740446 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c009d00 con 0x7ff2740ffe00 2024-05-08T22:05:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.258+0000 7ff272ffd700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 0x7ff274068da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff26400e970 tx=0x7ff26400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.259+0000 7ff25bfff700 1 -- 172.21.15.64:0/1930740446 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264009790 con 0x7ff2740ffe00 2024-05-08T22:05:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.259+0000 7ff25bfff700 1 -- 172.21.15.64:0/1930740446 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26400eec0 con 0x7ff2740ffe00 2024-05-08T22:05:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.259+0000 7ff25bfff700 1 -- 172.21.15.64:0/1930740446 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264003cf0 con 0x7ff2740ffe00 2024-05-08T22:05:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.261+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2741a1240 con 0x7ff2740ffe00 2024-05-08T22:05:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.261+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2741a16d0 con 0x7ff2740ffe00 2024-05-08T22:05:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.264+0000 7ff25bfff700 1 -- 172.21.15.64:0/1930740446 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff26400ccd0 con 0x7ff2740ffe00 2024-05-08T22:05:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.265+0000 7ff25bfff700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff25c0a11a0 0x7ff25c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.265+0000 7ff25bfff700 1 -- 172.21.15.64:0/1930740446 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff264014070 con 0x7ff2740ffe00 2024-05-08T22:05:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.265+0000 7ff2727fc700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff25c0a11a0 0x7ff25c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.266+0000 7ff28590bb80 1 Processor -- start 2024-05-08T22:05:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.266+0000 7ff2727fc700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff25c0a11a0 0x7ff25c0a3660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff274069360 tx=0x7ff268009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.266+0000 7ff28590bb80 1 -- start start 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.266+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df6a80 0x55a2f4df6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.267+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.267+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4df8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.267+0000 7ff28590bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4d4e0f0 con 0x55a2f4df82d0 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.267+0000 7ff28590bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4d4bab0 con 0x55a2f4df73e0 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.267+0000 7ff28590bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4d473a0 con 0x55a2f4df6a80 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.267+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40212/0 (socket says 172.21.15.64:40212) 2024-05-08T22:05:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 -- 172.21.15.64:0/3055052594 learned_addr learned my addr 172.21.15.64:0/3055052594 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 -- 172.21.15.64:0/3055052594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df6a80 msgr2=0x55a2f4df6ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:05:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df6a80 0x55a2f4df6ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 -- 172.21.15.64:0/3055052594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 msgr2=0x55a2f4df8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4df8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 -- 172.21.15.64:0/3055052594 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f4bc8350 con 0x55a2f4df73e0 2024-05-08T22:05:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2727fc700 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df7cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff268011e10 tx=0x7ff268014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354dd7233d9814bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.268+0000 7ff2597fa700 1 -- 172.21.15.64:0/3055052594 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26801f420 con 0x55a2f4df73e0 2024-05-08T22:05:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff2597fa700 1 -- 172.21.15.64:0/3055052594 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff268016070 con 0x55a2f4df73e0 2024-05-08T22:05:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff2597fa700 1 -- 172.21.15.64:0/3055052594 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26801d970 con 0x55a2f4df73e0 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 -- 172.21.15.64:0/3055052594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 msgr2=0x55a2f4df7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df7cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff268011e10 tx=0x7ff268014000 comp rx=0 tx=0).stop 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 -- 172.21.15.64:0/3055052594 shutdown_connections 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4df8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df6a80 0x55a2f4df6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 --2- 172.21.15.64:0/3055052594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df7cf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.269+0000 7ff28590bb80 1 -- 172.21.15.64:0/3055052594 >> 172.21.15.64:0/3055052594 conn(0x55a2f4d551b0 msgr2=0x55a2f4df3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.270+0000 7ff28590bb80 1 -- 172.21.15.64:0/3055052594 shutdown_connections 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.270+0000 7ff28590bb80 1 -- 172.21.15.64:0/3055052594 wait complete. 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.270+0000 7ff28590bb80 1 -- start start 2024-05-08T22:05:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.270+0000 7ff28590bb80 1 Processor -- start 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 0x55a2f4e00a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df82d0 0x55a2f4e00fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4ce3500 con 0x55a2f4e05fe0 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4ce31a0 con 0x55a2f4df82d0 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4bca640 con 0x55a2f4df73e0 2024-05-08T22:05:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff28590bb80 10 client.0 _ll_register_callbacks cb 0x55a2f4cecad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff2737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff2737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44456/0 (socket says 172.21.15.64:44456) 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.271+0000 7ff2737fe700 1 -- 172.21.15.64:0/606255083 learned_addr learned my addr 172.21.15.64:0/606255083 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff2737fe700 1 -- 172.21.15.64:0/606255083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 msgr2=0x55a2f4e00a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff2737fe700 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 0x55a2f4e00a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff2737fe700 1 -- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df82d0 msgr2=0x55a2f4e00fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff2737fe700 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df82d0 0x55a2f4e00fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff2737fe700 1 -- 172.21.15.64:0/606255083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff268011af0 con 0x55a2f4e05fe0 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff2737fe700 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e083d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009a20 tx=0x7ff26c008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a59793d66e9ee25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.272+0000 7ff256ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:05:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff256ff5700 1 -- 172.21.15.64:0/606255083 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00e040 con 0x55a2f4e05fe0 2024-05-08T22:05:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff256ff5700 1 -- 172.21.15.64:0/606255083 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff26c00baa0 con 0x55a2f4e05fe0 2024-05-08T22:05:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2f4ca0210 con 0x55a2f4e05fe0 2024-05-08T22:05:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff256ff5700 1 -- 172.21.15.64:0/606255083 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c02a030 con 0x55a2f4e05fe0 2024-05-08T22:05:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 10 client.15996 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55a2f4b74b00 con 0x55a2f4e05fe0 2024-05-08T22:05:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff256ff5700 1 -- 172.21.15.64:0/606255083 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2f4ca0210 con 0x55a2f4e05fe0 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff256ff5700 1 -- 172.21.15.64:0/606255083 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7ff26c022020 con 0x55a2f4e05fe0 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 10 client.15996 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 20 client.15996 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e154a0 0x55a2f4e178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 4 client.15996 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.273+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2f4dc67b0 con 0x55a2f4e154a0 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.274+0000 7ff272ffd700 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e154a0 0x55a2f4e178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.274+0000 7ff256ff5700 10 client.15996 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.274+0000 7ff272ffd700 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e154a0 0x55a2f4e178c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eab285cf322a996 server_cookie=e0251fcbca03882 in_seq=0 out_seq=0 2024-05-08T22:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.275+0000 7ff256ff5700 1 -- 172.21.15.64:0/606255083 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2f4dc67b0 con 0x55a2f4e154a0 2024-05-08T22:05:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.275+0000 7ff256ff5700 10 client.15996 handle_command_reply: tid=0 2024-05-08T22:05:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.275+0000 7ff28590bb80 1 client.15996 shutdown 2024-05-08T22:05:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.277+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 msgr2=0x55a2f4e083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.277+0000 7ff28590bb80 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e083d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009a20 tx=0x7ff26c008ef0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.277+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 shutdown_connections 2024-05-08T22:05:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e154a0 0x55a2f4e178c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e083d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 0x55a2f4e00a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 --2- 172.21.15.64:0/606255083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df82d0 0x55a2f4e00fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 >> 172.21.15.64:0/606255083 conn(0x55a2f4d551b0 msgr2=0x55a2f4df1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff256ff5700 0 client.15996 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 shutdown_connections 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 1 -- 172.21.15.64:0/606255083 wait complete. 2024-05-08T22:05:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.278+0000 7ff28590bb80 20 client.15996 trim_cache size 0 max 16384 2024-05-08T22:05:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 Processor -- start 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 -- start start 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4dfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4e05fe0 0x55a2f4dff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4dc67b0 con 0x55a2f4df82d0 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4ca0210 con 0x55a2f4df73e0 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.302+0000 7ff28590bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4df2af0 con 0x55a2f4e05fe0 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2737fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4e05fe0 0x55a2f4dff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4dfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4dfea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44462/0 (socket says 172.21.15.64:44462) 2024-05-08T22:05:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 -- 172.21.15.64:0/119777291 learned_addr learned my addr 172.21.15.64:0/119777291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 -- 172.21.15.64:0/119777291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4e05fe0 msgr2=0x55a2f4dff530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4e05fe0 0x55a2f4dff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 -- 172.21.15.64:0/119777291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 msgr2=0x55a2f4df86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.303+0000 7ff2727fc700 1 -- 172.21.15.64:0/119777291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f4d0d500 con 0x55a2f4df82d0 2024-05-08T22:05:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.304+0000 7ff2727fc700 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4dfea20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff268005720 tx=0x7ff268005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7af37f66276fe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.305+0000 7ff259ffb700 1 -- 172.21.15.64:0/119777291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26802c070 con 0x55a2f4df82d0 2024-05-08T22:05:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.305+0000 7ff259ffb700 1 -- 172.21.15.64:0/119777291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2680044e0 con 0x55a2f4df82d0 2024-05-08T22:05:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.305+0000 7ff259ffb700 1 -- 172.21.15.64:0/119777291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26801f680 con 0x55a2f4df82d0 2024-05-08T22:05:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.306+0000 7ff28590bb80 1 -- 172.21.15.64:0/119777291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 msgr2=0x55a2f4dfea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.306+0000 7ff28590bb80 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4dfea20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff268005720 tx=0x7ff268005750 comp rx=0 tx=0).stop 2024-05-08T22:05:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.315+0000 7ff28590bb80 1 -- 172.21.15.64:0/119777291 shutdown_connections 2024-05-08T22:05:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.315+0000 7ff28590bb80 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4df73e0 0x55a2f4df86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4e05fe0 0x55a2f4dff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 --2- 172.21.15.64:0/119777291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4dfea20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 -- 172.21.15.64:0/119777291 >> 172.21.15.64:0/119777291 conn(0x55a2f4d551b0 msgr2=0x55a2f4dfff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 -- 172.21.15.64:0/119777291 shutdown_connections 2024-05-08T22:05:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 -- 172.21.15.64:0/119777291 wait complete. 2024-05-08T22:05:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 -- start start 2024-05-08T22:05:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.316+0000 7ff28590bb80 1 Processor -- start 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 0x55a2f4e67600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4e25ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4e7b470 con 0x55a2f4df82d0 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4e7b5b0 con 0x55a2f4e05fe0 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f4e7b770 con 0x55a2f4df73e0 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff28590bb80 10 client.0 _ll_register_callbacks cb 0x55a2f4cecad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4e25ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4e25ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44478/0 (socket says 172.21.15.64:44478) 2024-05-08T22:05:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff2727fc700 1 -- 172.21.15.64:0/51692776 learned_addr learned my addr 172.21.15.64:0/51692776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff2727fc700 1 -- 172.21.15.64:0/51692776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 msgr2=0x55a2f4e67600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.317+0000 7ff2727fc700 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 0x55a2f4e67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.318+0000 7ff2727fc700 1 -- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4e05fe0 msgr2=0x55a2f4e67c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.318+0000 7ff2727fc700 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.318+0000 7ff2727fc700 1 -- 172.21.15.64:0/51692776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2f4bc8350 con 0x55a2f4df82d0 2024-05-08T22:05:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.318+0000 7ff2727fc700 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4e25ce0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff2680097d0 tx=0x7ff268005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c636eae566d956 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.324+0000 7ff256ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:05:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.324+0000 7ff256ff5700 1 -- 172.21.15.64:0/51692776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26802c050 con 0x55a2f4df82d0 2024-05-08T22:05:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.324+0000 7ff256ff5700 1 -- 172.21.15.64:0/51692776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff268018070 con 0x55a2f4df82d0 2024-05-08T22:05:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.324+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff268016070 con 0x55a2f4df82d0 2024-05-08T22:05:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.333+0000 7ff256ff5700 1 -- 172.21.15.64:0/51692776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26801f630 con 0x55a2f4df82d0 2024-05-08T22:05:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.340+0000 7ff28590bb80 10 client.16002 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.340+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55a2f4bccc70 con 0x55a2f4df82d0 2024-05-08T22:05:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.343+0000 7ff256ff5700 1 -- 172.21.15.64:0/51692776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff268016070 con 0x55a2f4df82d0 2024-05-08T22:05:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.343+0000 7ff256ff5700 1 -- 172.21.15.64:0/51692776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7ff268030a90 con 0x55a2f4df82d0 2024-05-08T22:05:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.343+0000 7ff28590bb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.344+0000 7ff28590bb80 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.344+0000 7ff28590bb80 20 client.16002 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.344+0000 7ff28590bb80 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e7cbf0 0x55a2f4e7efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.344+0000 7ff28590bb80 4 client.16002 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:05:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.344+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a2f4d3fe60 con 0x55a2f4e7cbf0 2024-05-08T22:05:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.352+0000 7ff272ffd700 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e7cbf0 0x55a2f4e7efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.356+0000 7ff256ff5700 10 client.16002 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.356+0000 7ff272ffd700 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e7cbf0 0x55a2f4e7efe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a643548e860475 server_cookie=757e532b9b150dd6 in_seq=0 out_seq=0 2024-05-08T22:05:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.423+0000 7ff256ff5700 1 -- 172.21.15.64:0/51692776 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a2f4d3fe60 con 0x55a2f4e7cbf0 2024-05-08T22:05:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.423+0000 7ff256ff5700 10 client.16002 handle_command_reply: tid=0 2024-05-08T22:05:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.423+0000 7ff28590bb80 1 client.16002 shutdown 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 msgr2=0x55a2f4e25ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4e25ce0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff2680097d0 tx=0x7ff268005d50 comp rx=0 tx=0).stop 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 shutdown_connections 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2f4e05fe0 0x55a2f4e67c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55a2f4e7cbf0 0x55a2f4e7efe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2f4df73e0 0x55a2f4e67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 --2- 172.21.15.64:0/51692776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2f4df82d0 0x55a2f4e25ce0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 >> 172.21.15.64:0/51692776 conn(0x55a2f4d551b0 msgr2=0x55a2f4dfff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff256ff5700 0 client.16002 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 shutdown_connections 2024-05-08T22:05:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.426+0000 7ff28590bb80 1 -- 172.21.15.64:0/51692776 wait complete. 2024-05-08T22:05:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.429+0000 7ff28590bb80 20 client.16002 trim_cache size 0 max 16384 2024-05-08T22:05:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:05:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:05:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:05:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:05:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff25c0a11a0 msgr2=0x7ff25c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff25c0a11a0 0x7ff25c0a3660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff274069360 tx=0x7ff268009260 comp rx=0 tx=0).stop 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 msgr2=0x7ff274068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 0x7ff274068da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff26400e970 tx=0x7ff26400ec80 comp rx=0 tx=0).stop 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 shutdown_connections 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7ff25c0a11a0 0x7ff25c0a3660 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2740ffe00 0x7ff274068da0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff274101990 0x7ff2741a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 --2- 172.21.15.64:0/1930740446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff274100ff0 0x7ff27419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.430+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 >> 172.21.15.64:0/1930740446 conn(0x7ff2740fb390 msgr2=0x7ff2741042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.431+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 shutdown_connections 2024-05-08T22:05:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:41.432+0000 7ff279751700 1 -- 172.21.15.64:0/1930740446 wait complete. 2024-05-08T22:05:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 650.0 s, 1448.2 tps, lat 68.886 ms stddev 83.265 2024-05-08T22:05:42.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:42 smithi186 ceph-mon[46619]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-05-08T22:05:42.978 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:42 smithi064 ceph-mon[42500]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-05-08T22:05:43.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:42 smithi160 ceph-mon[46962]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-05-08T22:05:44.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:44 smithi186 ceph-mon[46619]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 MiB/s wr, 710 op/s 2024-05-08T22:05:44.971 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:44 smithi064 ceph-mon[42500]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 MiB/s wr, 710 op/s 2024-05-08T22:05:45.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:44 smithi160 ceph-mon[46962]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 MiB/s wr, 710 op/s 2024-05-08T22:05:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 655.0 s, 1142.8 tps, lat 87.061 ms stddev 96.829 2024-05-08T22:05:46.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:46 smithi186 ceph-mon[46619]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 890 op/s 2024-05-08T22:05:46.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:46 smithi064 ceph-mon[42500]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 890 op/s 2024-05-08T22:05:47.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:46 smithi160 ceph-mon[46962]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 890 op/s 2024-05-08T22:05:48.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:48 smithi186 ceph-mon[46619]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-08T22:05:48.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:48 smithi064 ceph-mon[42500]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-08T22:05:49.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:48 smithi160 ceph-mon[46962]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-08T22:05:51.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 660.0 s, 1605.0 tps, lat 62.998 ms stddev 69.835 2024-05-08T22:05:51.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:51 smithi186 ceph-mon[46619]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 875 op/s 2024-05-08T22:05:51.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:51 smithi064 ceph-mon[42500]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 875 op/s 2024-05-08T22:05:52.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:51 smithi160 ceph-mon[46962]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 875 op/s 2024-05-08T22:05:53.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:53 smithi186 ceph-mon[46619]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 874 op/s 2024-05-08T22:05:53.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:53 smithi064 ceph-mon[42500]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 874 op/s 2024-05-08T22:05:54.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:53 smithi160 ceph-mon[46962]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 874 op/s 2024-05-08T22:05:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 -- 172.21.15.64:0/3064557561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7300ff580 msgr2=0x7fa7300ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 --2- 172.21.15.64:0/3064557561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7300ff580 0x7fa7300ff9a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa72000aa30 tx=0x7fa72000ad40 comp rx=0 tx=0).stop 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 -- 172.21.15.64:0/3064557561 shutdown_connections 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 --2- 172.21.15.64:0/3064557561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7300ff580 0x7fa7300ff9a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 --2- 172.21.15.64:0/3064557561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730101110 0x7fa7301079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 --2- 172.21.15.64:0/3064557561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa730100770 0x7fa730100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 -- 172.21.15.64:0/3064557561 >> 172.21.15.64:0/3064557561 conn(0x7fa7300faaf0 msgr2=0x7fa7300fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 -- 172.21.15.64:0/3064557561 shutdown_connections 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.581+0000 7fa73654d700 1 -- 172.21.15.64:0/3064557561 wait complete. 2024-05-08T22:05:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.582+0000 7fa73654d700 1 Processor -- start 2024-05-08T22:05:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.582+0000 7fa73654d700 1 -- start start 2024-05-08T22:05:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa73654d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7300ff580 0x7fa730196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa73654d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730100770 0x7fa7301973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa73654d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101110 0x7fa73019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa73654d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73019e9c0 con 0x7fa730100770 2024-05-08T22:05:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa73654d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73019eb00 con 0x7fa730101110 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa73654d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73019ec90 con 0x7fa7300ff580 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730100770 0x7fa7301973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730100770 0x7fa7301973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51752/0 (socket says 172.21.15.64:51752) 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa72f7fe700 1 -- 172.21.15.64:0/1998341691 learned_addr learned my addr 172.21.15.64:0/1998341691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa72ffff700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7300ff580 0x7fa730196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa734aea700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101110 0x7fa73019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa734aea700 1 -- 172.21.15.64:0/1998341691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7300ff580 msgr2=0x7fa730196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa734aea700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7300ff580 0x7fa730196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa734aea700 1 -- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730100770 msgr2=0x7fa7301973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa734aea700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730100770 0x7fa7301973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.583+0000 7fa734aea700 1 -- 172.21.15.64:0/1998341691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728009d00 con 0x7fa730101110 2024-05-08T22:05:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.584+0000 7fa734aea700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101110 0x7fa73019e3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa728009cd0 tx=0x7fa72800c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.584+0000 7fa72d7fa700 1 -- 172.21.15.64:0/1998341691 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728017070 con 0x7fa730101110 2024-05-08T22:05:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.584+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72000a630 con 0x7fa730101110 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.584+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa730194ae0 con 0x7fa730101110 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.584+0000 7fa72d7fa700 1 -- 172.21.15.64:0/1998341691 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72800f040 con 0x7fa730101110 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.584+0000 7fa72d7fa700 1 -- 172.21.15.64:0/1998341691 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72800e050 con 0x7fa730101110 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.586+0000 7fa72d7fa700 1 -- 172.21.15.64:0/1998341691 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa728003a00 con 0x7fa730101110 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.587+0000 7fa72d7fa700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7180a11a0 0x7fa7180a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.587+0000 7fa72d7fa700 1 -- 172.21.15.64:0/1998341691 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7280cc180 con 0x7fa730101110 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.587+0000 7fa72ffff700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7180a11a0 0x7fa7180a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.588+0000 7fa72ffff700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7180a11a0 0x7fa7180a3660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa720018040 tx=0x7fa720005980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 Processor -- start 2024-05-08T22:05:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 -- start start 2024-05-08T22:05:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd82f0 0x55cfb9fd8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9fd95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9f320f0 con 0x55cfb9fd9b40 2024-05-08T22:05:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9f2fab0 con 0x55cfb9fd8c50 2024-05-08T22:05:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.589+0000 7fa742707b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9f2b3a0 con 0x55cfb9fd82f0 2024-05-08T22:05:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.601+0000 7fa734aea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.601+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9fd95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.601+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9fd95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54440/0 (socket says 172.21.15.64:54440) 2024-05-08T22:05:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.601+0000 7fa734aea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51756/0 (socket says 172.21.15.64:51756) 2024-05-08T22:05:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.601+0000 7fa72f7fe700 1 -- 172.21.15.64:0/786537498 learned_addr learned my addr 172.21.15.64:0/786537498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.606+0000 7fa72ffff700 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd82f0 0x55cfb9fd8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.616+0000 7fa72f7fe700 1 -- 172.21.15.64:0/786537498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd82f0 msgr2=0x55cfb9fd8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.616+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd82f0 0x55cfb9fd8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.616+0000 7fa72f7fe700 1 -- 172.21.15.64:0/786537498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 msgr2=0x55cfb9fd9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.619+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.619+0000 7fa72f7fe700 1 -- 172.21.15.64:0/786537498 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfb9dac350 con 0x55cfb9fd8c50 2024-05-08T22:05:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.626+0000 7fa734aea700 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.634+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9fd95b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e50 tx=0x7fa724011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cad2f036927370 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.636+0000 7fa7167fc700 1 -- 172.21.15.64:0/786537498 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724004a30 con 0x55cfb9fd8c50 2024-05-08T22:05:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.636+0000 7fa7167fc700 1 -- 172.21.15.64:0/786537498 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7240054e0 con 0x55cfb9fd8c50 2024-05-08T22:05:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.636+0000 7fa7167fc700 1 -- 172.21.15.64:0/786537498 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724019930 con 0x55cfb9fd8c50 2024-05-08T22:05:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.639+0000 7fa742707b80 1 -- 172.21.15.64:0/786537498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 msgr2=0x55cfb9fd95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.639+0000 7fa742707b80 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9fd95b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e50 tx=0x7fa724011720 comp rx=0 tx=0).stop 2024-05-08T22:05:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 -- 172.21.15.64:0/786537498 shutdown_connections 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd82f0 0x55cfb9fd8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 --2- 172.21.15.64:0/786537498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9fd95b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 -- 172.21.15.64:0/786537498 >> 172.21.15.64:0/786537498 conn(0x55cfb9f391b0 msgr2=0x55cfb9fd7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 -- 172.21.15.64:0/786537498 shutdown_connections 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.644+0000 7fa742707b80 1 -- 172.21.15.64:0/786537498 wait complete. 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 -- start start 2024-05-08T22:05:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 Processor -- start 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ff58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ff5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 0x55cfb9ffce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9ec7500 con 0x55cfb9fd82f0 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9ec71a0 con 0x55cfb9fd8c50 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9dae640 con 0x55cfb9fd9b40 2024-05-08T22:05:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.645+0000 7fa742707b80 10 client.0 _ll_register_callbacks cb 0x55cfb9ed0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.646+0000 7fa72ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ff58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.646+0000 7fa72ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ff58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51758/0 (socket says 172.21.15.64:51758) 2024-05-08T22:05:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.646+0000 7fa72ffff700 1 -- 172.21.15.64:0/1753932998 learned_addr learned my addr 172.21.15.64:0/1753932998 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.646+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ff5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa72ffff700 1 -- 172.21.15.64:0/1753932998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 msgr2=0x55cfb9ffce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa72ffff700 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 0x55cfb9ffce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa72ffff700 1 -- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 msgr2=0x55cfb9ff5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa72ffff700 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ff5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa72ffff700 1 -- 172.21.15.64:0/1753932998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa724009a80 con 0x55cfb9fd82f0 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa72ffff700 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ff58e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa7301005d0 tx=0x7fa72000a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510042a58c2f4e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.649+0000 7fa70f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:05:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.650+0000 7fa70f7fe700 1 -- 172.21.15.64:0/1753932998 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720004b00 con 0x55cfb9fd82f0 2024-05-08T22:05:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.650+0000 7fa70f7fe700 1 -- 172.21.15.64:0/1753932998 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa720027040 con 0x55cfb9fd82f0 2024-05-08T22:05:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.651+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfb9e84210 con 0x55cfb9fd82f0 2024-05-08T22:05:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.651+0000 7fa70f7fe700 1 -- 172.21.15.64:0/1753932998 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa720004c60 con 0x55cfb9fd82f0 2024-05-08T22:05:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.651+0000 7fa742707b80 10 client.16014 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.651+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55cfb9d58b00 con 0x55cfb9fd82f0 2024-05-08T22:05:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa70f7fe700 1 -- 172.21.15.64:0/1753932998 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cfb9e84210 con 0x55cfb9fd82f0 2024-05-08T22:05:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa70f7fe700 1 -- 172.21.15.64:0/1753932998 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fa7200166b0 con 0x55cfb9fd82f0 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa742707b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa742707b80 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa742707b80 20 client.16014 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa742707b80 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba012720 0x55cfba014b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa742707b80 4 client.16014 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfb9faa7b0 con 0x55cfba012720 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.652+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba012720 0x55cfba014b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.653+0000 7fa70f7fe700 10 client.16014 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.653+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba012720 0x55cfba014b40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94026a828535f496 server_cookie=c6a0c19ead8f6f8c in_seq=0 out_seq=0 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa70f7fe700 1 -- 172.21.15.64:0/1753932998 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfb9faa7b0 con 0x55cfba012720 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa70f7fe700 10 client.16014 handle_command_reply: tid=0 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 client.16014 shutdown 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 msgr2=0x55cfb9ff58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ff58e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa7301005d0 tx=0x7fa72000a370 comp rx=0 tx=0).stop 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 shutdown_connections 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba012720 0x55cfba014b40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 0x55cfb9ffce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa70f7fe700 0 client.16014 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ff58e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 --2- 172.21.15.64:0/1753932998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ff5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 >> 172.21.15.64:0/1753932998 conn(0x55cfb9f391b0 msgr2=0x55cfb9fd5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 shutdown_connections 2024-05-08T22:05:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.654+0000 7fa742707b80 1 -- 172.21.15.64:0/1753932998 wait complete. 2024-05-08T22:05:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.655+0000 7fa742707b80 20 client.16014 trim_cache size 0 max 16384 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 Processor -- start 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 -- start start 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfba014b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ffcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9fd0330 con 0x55cfb9fd82f0 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9fdec80 con 0x55cfb9fd8c50 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa742707b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb9fd7f20 con 0x55cfb9fd9b40 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ffcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ffcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54454/0 (socket says 172.21.15.64:54454) 2024-05-08T22:05:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa72f7fe700 1 -- 172.21.15.64:0/235115563 learned_addr learned my addr 172.21.15.64:0/235115563 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.680+0000 7fa72ffff700 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfba014b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa72f7fe700 1 -- 172.21.15.64:0/235115563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 msgr2=0x55cfb9fd9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa72f7fe700 1 -- 172.21.15.64:0/235115563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 msgr2=0x55cfba014b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfba014b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa72f7fe700 1 -- 172.21.15.64:0/235115563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfb9ef1500 con 0x55cfb9fd8c50 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa72f7fe700 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ffcdf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa724009310 tx=0x7fa7240057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f825c90e90d25a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa716ffd700 1 -- 172.21.15.64:0/235115563 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724029070 con 0x55cfb9fd8c50 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa716ffd700 1 -- 172.21.15.64:0/235115563 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa724004500 con 0x55cfb9fd8c50 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.681+0000 7fa716ffd700 1 -- 172.21.15.64:0/235115563 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724019430 con 0x55cfb9fd8c50 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.682+0000 7fa742707b80 1 -- 172.21.15.64:0/235115563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 msgr2=0x55cfb9ffcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.682+0000 7fa742707b80 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ffcdf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa724009310 tx=0x7fa7240057d0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.686+0000 7fa742707b80 1 -- 172.21.15.64:0/235115563 shutdown_connections 2024-05-08T22:05:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.686+0000 7fa742707b80 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd9b40 0x55cfb9fd9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.686+0000 7fa742707b80 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfba014b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.686+0000 7fa742707b80 1 --2- 172.21.15.64:0/235115563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd8c50 0x55cfb9ffcdf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.686+0000 7fa742707b80 1 -- 172.21.15.64:0/235115563 >> 172.21.15.64:0/235115563 conn(0x55cfb9f391b0 msgr2=0x55cfb9ff4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.691+0000 7fa742707b80 1 -- 172.21.15.64:0/235115563 shutdown_connections 2024-05-08T22:05:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.696+0000 7fa742707b80 1 -- 172.21.15.64:0/235115563 wait complete. 2024-05-08T22:05:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.696+0000 7fa742707b80 1 -- start start 2024-05-08T22:05:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.696+0000 7fa742707b80 1 Processor -- start 2024-05-08T22:05:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ffade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd8c50 0x55cfba064bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 0x55cfba067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfba067680 con 0x55cfb9fd82f0 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfba0677c0 con 0x55cfb9fd9b40 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfba067980 con 0x55cfb9fd8c50 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa742707b80 10 client.0 _ll_register_callbacks cb 0x55cfb9ed0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 0x55cfba067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 0x55cfba067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:54466/0 (socket says 172.21.15.64:54466) 2024-05-08T22:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 -- 172.21.15.64:0/3005621985 learned_addr learned my addr 172.21.15.64:0/3005621985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:05:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 -- 172.21.15.64:0/3005621985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd8c50 msgr2=0x55cfba064bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:05:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd8c50 0x55cfba064bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 -- 172.21.15.64:0/3005621985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 msgr2=0x55cfb9ffade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ffade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.700+0000 7fa734aea700 1 -- 172.21.15.64:0/3005621985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfb9dac350 con 0x55cfb9fd9b40 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.701+0000 7fa734aea700 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 0x55cfba067140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7280c9c30 tx=0x7fa728084390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeddce978cd90572 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.701+0000 7fa70f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.701+0000 7fa70f7fe700 1 -- 172.21.15.64:0/3005621985 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728098c70 con 0x55cfb9fd9b40 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.701+0000 7fa70f7fe700 1 -- 172.21.15.64:0/3005621985 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa728095040 con 0x55cfb9fd9b40 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.702+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7240054e0 con 0x55cfb9fd9b40 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.702+0000 7fa70f7fe700 1 -- 172.21.15.64:0/3005621985 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7280b3070 con 0x55cfb9fd9b40 2024-05-08T22:05:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.702+0000 7fa742707b80 10 client.25799 fetch_fsmap learned FSMap version 45 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.702+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55cfb9db0c70 con 0x55cfb9fd9b40 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.702+0000 7fa70f7fe700 1 -- 172.21.15.64:0/3005621985 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7240054e0 con 0x55cfb9fd9b40 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.703+0000 7fa70f7fe700 1 -- 172.21.15.64:0/3005621985 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fa7280a1570 con 0x55cfb9fd9b40 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.703+0000 7fa742707b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.703+0000 7fa742707b80 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.703+0000 7fa742707b80 20 client.25799 populate_metadata read hostname 'smithi064' 2024-05-08T22:05:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.704+0000 7fa742707b80 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba069400 0x55cfba06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.704+0000 7fa742707b80 4 client.25799 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:05:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.704+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cfb9f23e60 con 0x55cfba069400 2024-05-08T22:05:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.704+0000 7fa72ffff700 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba069400 0x55cfba06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.704+0000 7fa70f7fe700 10 client.25799 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.704+0000 7fa72ffff700 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba069400 0x55cfba06b820 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a2b3666ad60d8c server_cookie=b25414db92fd1e0f in_seq=0 out_seq=0 2024-05-08T22:05:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa70f7fe700 1 -- 172.21.15.64:0/3005621985 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cfb9f23e60 con 0x55cfba069400 2024-05-08T22:05:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa70f7fe700 10 client.25799 handle_command_reply: tid=0 2024-05-08T22:05:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 client.25799 shutdown 2024-05-08T22:05:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 msgr2=0x55cfba067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 0x55cfba067140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7280c9c30 tx=0x7fa728084390 comp rx=0 tx=0).stop 2024-05-08T22:05:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 shutdown_connections 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cfb9fd9b40 0x55cfba067140 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55cfba069400 0x55cfba06b820 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cfb9fd82f0 0x55cfb9ffade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 --2- 172.21.15.64:0/3005621985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cfb9fd8c50 0x55cfba064bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa70f7fe700 0 client.25799 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 >> 172.21.15.64:0/3005621985 conn(0x55cfb9f391b0 msgr2=0x55cfb9ff3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 shutdown_connections 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.713+0000 7fa742707b80 1 -- 172.21.15.64:0/3005621985 wait complete. 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa742707b80 20 client.25799 trim_cache size 0 max 16384 2024-05-08T22:05:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:05:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:05:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:05:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:05:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7180a11a0 msgr2=0x7fa7180a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7180a11a0 0x7fa7180a3660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa720018040 tx=0x7fa720005980 comp rx=0 tx=0).stop 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101110 msgr2=0x7fa73019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101110 0x7fa73019e3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa728009cd0 tx=0x7fa72800c620 comp rx=0 tx=0).stop 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 shutdown_connections 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7180a11a0 0x7fa7180a3660 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.714+0000 7fa73654d700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101110 0x7fa73019e3f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.715+0000 7fa73654d700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7300ff580 0x7fa730196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.715+0000 7fa73654d700 1 --2- 172.21.15.64:0/1998341691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa730100770 0x7fa7301973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.715+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 >> 172.21.15.64:0/1998341691 conn(0x7fa7300faaf0 msgr2=0x7fa730103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.715+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 shutdown_connections 2024-05-08T22:05:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:05:54.715+0000 7fa73654d700 1 -- 172.21.15.64:0/1998341691 wait complete. 2024-05-08T22:05:55.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:55 smithi186 ceph-mon[46619]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-05-08T22:05:55.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:55 smithi064 ceph-mon[42500]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-05-08T22:05:56.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:55 smithi160 ceph-mon[46962]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-05-08T22:05:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 665.0 s, 1683.4 tps, lat 59.145 ms stddev 65.403 2024-05-08T22:05:56.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:56 smithi186 ceph-mon[46619]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 833 op/s 2024-05-08T22:05:56.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:56 smithi064 ceph-mon[42500]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 833 op/s 2024-05-08T22:05:57.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:56 smithi160 ceph-mon[46962]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 833 op/s 2024-05-08T22:05:59.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:05:59 smithi186 ceph-mon[46619]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 606 op/s 2024-05-08T22:05:59.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:05:59 smithi064 ceph-mon[42500]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 606 op/s 2024-05-08T22:06:00.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:05:59 smithi160 ceph-mon[46962]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 606 op/s 2024-05-08T22:06:00.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:00 smithi186 ceph-mon[46619]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 755 op/s 2024-05-08T22:06:00.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:00 smithi064 ceph-mon[42500]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 755 op/s 2024-05-08T22:06:01.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:00 smithi160 ceph-mon[46962]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 755 op/s 2024-05-08T22:06:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 670.0 s, 1458.8 tps, lat 69.011 ms stddev 82.629 2024-05-08T22:06:02.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:02 smithi064 ceph-mon[42500]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 732 op/s 2024-05-08T22:06:03.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:02 smithi160 ceph-mon[46962]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 732 op/s 2024-05-08T22:06:03.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:02 smithi186 ceph-mon[46619]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 732 op/s 2024-05-08T22:06:05.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:05 smithi186 ceph-mon[46619]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-05-08T22:06:05.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:05 smithi064 ceph-mon[42500]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-05-08T22:06:06.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:05 smithi160 ceph-mon[46962]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-05-08T22:06:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 675.0 s, 1231.4 tps, lat 80.221 ms stddev 138.737 2024-05-08T22:06:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 -- 172.21.15.64:0/1914302593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e41000c0 msgr2=0x7fd2e4100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 --2- 172.21.15.64:0/1914302593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e41000c0 0x7fd2e4100520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009990 tx=0x7fd2e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 -- 172.21.15.64:0/1914302593 shutdown_connections 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 --2- 172.21.15.64:0/1914302593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e4100a60 0x7fd2e4108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 --2- 172.21.15.64:0/1914302593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2e4101390 0x7fd2e41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 --2- 172.21.15.64:0/1914302593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e41000c0 0x7fd2e4100520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 -- 172.21.15.64:0/1914302593 >> 172.21.15.64:0/1914302593 conn(0x7fd2e40faaf0 msgr2=0x7fd2e40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 -- 172.21.15.64:0/1914302593 shutdown_connections 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.886+0000 7fd2ec249700 1 -- 172.21.15.64:0/1914302593 wait complete. 2024-05-08T22:06:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 Processor -- start 2024-05-08T22:06:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 -- start start 2024-05-08T22:06:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2e41000c0 0x7fd2e406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e4100a60 0x7fd2e406fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 0x7fd2e41a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4072e50 con 0x7fd2e4100a60 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4072fc0 con 0x7fd2e4101390 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ec249700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e41a5420 con 0x7fd2e41000c0 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ea7e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 0x7fd2e41a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2e9fe5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2e41000c0 0x7fd2e406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ea7e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 0x7fd2e41a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:48716/0 (socket says 172.21.15.64:48716) 2024-05-08T22:06:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.887+0000 7fd2ea7e6700 1 -- 172.21.15.64:0/2257739574 learned_addr learned my addr 172.21.15.64:0/2257739574 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:07.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:07 smithi186 ceph-mon[46619]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 1.73k op/s 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.892+0000 7fd2ea7e6700 1 -- 172.21.15.64:0/2257739574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2e41000c0 msgr2=0x7fd2e406f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.892+0000 7fd2ea7e6700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2e41000c0 0x7fd2e406f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.892+0000 7fd2ea7e6700 1 -- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e4100a60 msgr2=0x7fd2e406fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.892+0000 7fd2ea7e6700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e4100a60 0x7fd2e406fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.892+0000 7fd2ea7e6700 1 -- 172.21.15.64:0/2257739574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00095f0 con 0x7fd2e4101390 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.892+0000 7fd2ea7e6700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 0x7fd2e41a4ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00b970 tx=0x7fd2dc00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.896+0000 7fd2daffd700 1 -- 172.21.15.64:0/2257739574 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00c9b0 con 0x7fd2e4101390 2024-05-08T22:06:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.896+0000 7fd2daffd700 1 -- 172.21.15.64:0/2257739574 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2dc014440 con 0x7fd2e4101390 2024-05-08T22:06:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.896+0000 7fd2daffd700 1 -- 172.21.15.64:0/2257739574 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0075d0 con 0x7fd2e4101390 2024-05-08T22:06:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.896+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e41ade90 con 0x7fd2e4101390 2024-05-08T22:06:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.896+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e41ae340 con 0x7fd2e4101390 2024-05-08T22:06:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.901+0000 7fd2f8403b80 1 Processor -- start 2024-05-08T22:06:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2daffd700 1 -- 172.21.15.64:0/2257739574 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd2dc0145b0 con 0x7fd2e4101390 2024-05-08T22:06:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 -- start start 2024-05-08T22:06:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966268ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966269cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x55796626a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579661c00f0 con 0x557966268a80 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579661bdab0 con 0x5579662693e0 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579661b93a0 con 0x55796626a2d0 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2e9fe5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966269cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2e9fe5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966269cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:48728/0 (socket says 172.21.15.64:48728) 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/620071518 learned_addr learned my addr 172.21.15.64:0/620071518 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.902+0000 7fd2daffd700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2d00a1250 0x7fd2d00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2daffd700 1 -- 172.21.15.64:0/2257739574 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2dc006e40 con 0x7fd2e4101390 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/620071518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 msgr2=0x55796626a750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x55796626a750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/620071518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 msgr2=0x557966268ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966268ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/620071518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55796603a350 con 0x5579662693e0 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2ea7e6700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2d00a1250 0x7fd2d00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966269cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009860 tx=0x7fd2d4009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f3a930bc0572fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.903+0000 7fd2cffff700 1 -- 172.21.15.64:0/620071518 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400d610 con 0x5579662693e0 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2cffff700 1 -- 172.21.15.64:0/620071518 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d400dc30 con 0x5579662693e0 2024-05-08T22:06:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2cffff700 1 -- 172.21.15.64:0/620071518 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4014d00 con 0x5579662693e0 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 -- 172.21.15.64:0/620071518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 msgr2=0x557966269cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2ea7e6700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2d00a1250 0x7fd2d00a3710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc014880 tx=0x7fd2dc004be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966269cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009860 tx=0x7fd2d4009b70 comp rx=0 tx=0).stop 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 -- 172.21.15.64:0/620071518 shutdown_connections 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x55796626a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966268ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/620071518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966269cf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.904+0000 7fd2f8403b80 1 -- 172.21.15.64:0/620071518 >> 172.21.15.64:0/620071518 conn(0x5579661c71b0 msgr2=0x557966265440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.905+0000 7fd2f8403b80 1 -- 172.21.15.64:0/620071518 shutdown_connections 2024-05-08T22:06:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.905+0000 7fd2f8403b80 1 -- 172.21.15.64:0/620071518 wait complete. 2024-05-08T22:06:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.905+0000 7fd2f8403b80 1 -- start start 2024-05-08T22:06:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.905+0000 7fd2f8403b80 1 Processor -- start 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557966268a80 0x5579662727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966272d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 0x557966279d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557966155500 con 0x55796626a2d0 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579661551a0 con 0x557966268a80 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796603c640 con 0x5579662693e0 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.907+0000 7fd2f8403b80 10 client.0 _ll_register_callbacks cb 0x55796615ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 0x557966279d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 0x557966279d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48034/0 (socket says 172.21.15.64:48034) 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 -- 172.21.15.64:0/1259069928 learned_addr learned my addr 172.21.15.64:0/1259069928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557966268a80 0x5579662727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966272d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 -- 172.21.15.64:0/1259069928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 msgr2=0x557966272d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966272d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 -- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557966268a80 msgr2=0x5579662727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557966268a80 0x5579662727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 -- 172.21.15.64:0/1259069928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d400e040 con 0x55796626a2d0 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 0x557966279d70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000bff0 tx=0x7fd2e000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e13184776f408db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:06:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/1259069928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0003b60 con 0x55796626a2d0 2024-05-08T22:06:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.908+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/1259069928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2e0003cc0 con 0x55796626a2d0 2024-05-08T22:06:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557966112210 con 0x55796626a2d0 2024-05-08T22:06:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/1259069928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e0005a00 con 0x55796626a2d0 2024-05-08T22:06:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 10 client.16020 fetch_fsmap learned FSMap version 45 2024-05-08T22:06:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x557965fe6b00 con 0x55796626a2d0 2024-05-08T22:06:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/1259069928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557966112210 con 0x55796626a2d0 2024-05-08T22:06:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/1259069928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd2e0018a00 con 0x55796626a2d0 2024-05-08T22:06:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:06:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 20 client.16020 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55796627e530 0x557966280950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 4 client.16020 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579662387b0 con 0x55796627e530 2024-05-08T22:06:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.909+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55796627e530 0x557966280950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.910+0000 7fd2cd7fa700 10 client.16020 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.910+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55796627e530 0x557966280950 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e743180a2ed6e4f server_cookie=e9cf501a2bee377b in_seq=0 out_seq=0 2024-05-08T22:06:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.910+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/1259069928 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579662387b0 con 0x55796627e530 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.910+0000 7fd2cd7fa700 10 client.16020 handle_command_reply: tid=0 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.910+0000 7fd2f8403b80 1 client.16020 shutdown 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 msgr2=0x557966279d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 0x557966279d70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000bff0 tx=0x7fd2e000b540 comp rx=0 tx=0).stop 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 shutdown_connections 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557966268a80 0x5579662727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55796627e530 0x557966280950 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55796626a2d0 0x557966279d70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/1259069928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966272d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2cd7fa700 0 client.16020 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 >> 172.21.15.64:0/1259069928 conn(0x5579661c71b0 msgr2=0x557966263860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 shutdown_connections 2024-05-08T22:06:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 1 -- 172.21.15.64:0/1259069928 wait complete. 2024-05-08T22:06:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.911+0000 7fd2f8403b80 20 client.16020 trim_cache size 0 max 16384 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.931+0000 7fd2f8403b80 1 Processor -- start 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.931+0000 7fd2f8403b80 1 -- start start 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x55796627a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966279d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55796626a2d0 0x55796626a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579662609b0 con 0x557966268a80 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796627aef0 con 0x55796626a2d0 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557966271390 con 0x5579662693e0 2024-05-08T22:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x55796627a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x55796627a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48042/0 (socket says 172.21.15.64:48042) 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 -- 172.21.15.64:0/3987688087 learned_addr learned my addr 172.21.15.64:0/3987688087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966279d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55796626a2d0 0x55796626a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 -- 172.21.15.64:0/3987688087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 msgr2=0x557966279d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966279d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 -- 172.21.15.64:0/3987688087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55796626a2d0 msgr2=0x55796626a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55796626a2d0 0x55796626a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 -- 172.21.15.64:0/3987688087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55796617f500 con 0x557966268a80 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.932+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x55796627a420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00273f0 tx=0x7fd2e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ebc8e9dbf93373 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3987688087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0003b60 con 0x557966268a80 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3987688087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e0018ba0 con 0x557966268a80 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3987688087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0005600 con 0x557966268a80 2024-05-08T22:06:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3987688087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 msgr2=0x55796627a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x55796627a420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00273f0 tx=0x7fd2e0005fd0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3987688087 shutdown_connections 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55796626a2d0 0x55796626a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x55796627a420 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3987688087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579662693e0 0x557966279d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.933+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3987688087 >> 172.21.15.64:0/3987688087 conn(0x5579661c71b0 msgr2=0x557966271d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.934+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3987688087 shutdown_connections 2024-05-08T22:06:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.935+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3987688087 wait complete. 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.935+0000 7fd2f8403b80 1 -- start start 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.935+0000 7fd2f8403b80 1 Processor -- start 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966277340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966277920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x557966278660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557966278ba0 con 0x557966268a80 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557966278d10 con 0x5579662693e0 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557966278ed0 con 0x55796626a2d0 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2f8403b80 10 client.0 _ll_register_callbacks cb 0x55796615ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e97e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966277340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966277920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e97e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966277340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48044/0 (socket says 172.21.15.64:48044) 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966277920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:48760/0 (socket says 172.21.15.64:48760) 2024-05-08T22:06:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e97e4700 1 -- 172.21.15.64:0/3749695673 learned_addr learned my addr 172.21.15.64:0/3749695673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x557966278660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/3749695673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 msgr2=0x557966278660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x557966278660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/3749695673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 msgr2=0x557966277340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966277340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.936+0000 7fd2e9fe5700 1 -- 172.21.15.64:0/3749695673 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557965fe6b00 con 0x5579662693e0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2e9fe5700 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966277920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009830 tx=0x7fd2d400df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e5a2a3f09b02ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/3749695673 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4024070 con 0x5579662693e0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2d400dc30 con 0x5579662693e0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/3749695673 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2d4025d90 con 0x5579662693e0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/3749695673 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d400d610 con 0x5579662693e0 2024-05-08T22:06:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 10 client.25817 fetch_fsmap learned FSMap version 45 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55796603ec70 con 0x5579662693e0 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/3749695673 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d400dc30 con 0x5579662693e0 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/3749695673 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd2d4009890 con 0x5579662693e0 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 20 client.25817 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5579662d4fe0 0x5579662d7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 4 client.25817 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.937+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5579661b1e60 con 0x5579662d4fe0 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.938+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5579662d4fe0 0x5579662d7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.938+0000 7fd2e97e4700 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5579662d4fe0 0x5579662d7400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04453728a774e0 server_cookie=81bf92e7898f3e0d in_seq=0 out_seq=0 2024-05-08T22:06:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.938+0000 7fd2cd7fa700 10 client.25817 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2cd7fa700 1 -- 172.21.15.64:0/3749695673 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5579661b1e60 con 0x5579662d4fe0 2024-05-08T22:06:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2cd7fa700 10 client.25817 handle_command_reply: tid=0 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 client.25817 shutdown 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 msgr2=0x557966277920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966277920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009830 tx=0x7fd2d400df60 comp rx=0 tx=0).stop 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 shutdown_connections 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5579662d4fe0 0x5579662d7400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55796626a2d0 0x557966278660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557966268a80 0x557966277340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 --2- 172.21.15.64:0/3749695673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579662693e0 0x557966277920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2cd7fa700 0 client.25817 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 >> 172.21.15.64:0/3749695673 conn(0x5579661c71b0 msgr2=0x557966265020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 shutdown_connections 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 1 -- 172.21.15.64:0/3749695673 wait complete. 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.948+0000 7fd2f8403b80 20 client.25817 trim_cache size 0 max 16384 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:06:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2d00a1250 msgr2=0x7fd2d00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2d00a1250 0x7fd2d00a3710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc014880 tx=0x7fd2dc004be0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 msgr2=0x7fd2e41a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 0x7fd2e41a4ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00b970 tx=0x7fd2dc00bd30 comp rx=0 tx=0).stop 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 shutdown_connections 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd2d00a1250 0x7fd2d00a3710 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e4101390 0x7fd2e41a4ee0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2e41000c0 0x7fd2e406f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 --2- 172.21.15.64:0/2257739574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e4100a60 0x7fd2e406fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 >> 172.21.15.64:0/2257739574 conn(0x7fd2e40faaf0 msgr2=0x7fd2e4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 shutdown_connections 2024-05-08T22:06:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:07.949+0000 7fd2ec249700 1 -- 172.21.15.64:0/2257739574 wait complete. 2024-05-08T22:06:07.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:07 smithi064 ceph-mon[42500]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 1.73k op/s 2024-05-08T22:06:08.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:07 smithi160 ceph-mon[46962]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 1.73k op/s 2024-05-08T22:06:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:08 smithi064 ceph-mon[42500]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 MiB/s wr, 1.85k op/s 2024-05-08T22:06:09.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:08 smithi160 ceph-mon[46962]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 MiB/s wr, 1.85k op/s 2024-05-08T22:06:09.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:08 smithi186 ceph-mon[46619]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 MiB/s wr, 1.85k op/s 2024-05-08T22:06:11.479 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 680.0 s, 1705.2 tps, lat 58.951 ms stddev 65.001 2024-05-08T22:06:11.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:11 smithi186 ceph-mon[46619]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 2.00k op/s 2024-05-08T22:06:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:11 smithi064 ceph-mon[42500]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 2.00k op/s 2024-05-08T22:06:12.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:11 smithi160 ceph-mon[46962]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 2.00k op/s 2024-05-08T22:06:13.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:13 smithi186 ceph-mon[46619]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2024-05-08T22:06:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:13 smithi064 ceph-mon[42500]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2024-05-08T22:06:14.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:13 smithi160 ceph-mon[46962]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2024-05-08T22:06:14.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:14 smithi186 ceph-mon[46619]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2024-05-08T22:06:14.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:14 smithi064 ceph-mon[42500]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2024-05-08T22:06:15.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:14 smithi160 ceph-mon[46962]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2024-05-08T22:06:16.485 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 685.0 s, 1854.4 tps, lat 54.108 ms stddev 61.915 2024-05-08T22:06:17.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:17 smithi186 ceph-mon[46619]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2024-05-08T22:06:17.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:17 smithi064 ceph-mon[42500]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2024-05-08T22:06:18.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:17 smithi160 ceph-mon[46962]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2024-05-08T22:06:19.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:19 smithi186 ceph-mon[46619]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 874 op/s 2024-05-08T22:06:19.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:19 smithi064 ceph-mon[42500]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 874 op/s 2024-05-08T22:06:20.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:19 smithi160 ceph-mon[46962]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 874 op/s 2024-05-08T22:06:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 -- 172.21.15.64:0/3823084661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b8073a00 msgr2=0x7f52b8073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 --2- 172.21.15.64:0/3823084661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b8073a00 0x7f52b8073e80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009f90 tx=0x7f52b000b330 comp rx=0 tx=0).stop 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 -- 172.21.15.64:0/3823084661 shutdown_connections 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 --2- 172.21.15.64:0/3823084661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b810d4c0 0x7f52b810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 --2- 172.21.15.64:0/3823084661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 0x7f52b8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 --2- 172.21.15.64:0/3823084661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b8073a00 0x7f52b8073e80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 -- 172.21.15.64:0/3823084661 >> 172.21.15.64:0/3823084661 conn(0x7f52b80fc460 msgr2=0x7f52b80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 -- 172.21.15.64:0/3823084661 shutdown_connections 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.094+0000 7f52bd443700 1 -- 172.21.15.64:0/3823084661 wait complete. 2024-05-08T22:06:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 Processor -- start 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 -- start start 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8073a00 0x7f52b819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 0x7f52b819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 0x7f52b81a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81a51c0 con 0x7f52b810d4c0 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81a5300 con 0x7f52b8073a00 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52bd443700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81a5490 con 0x7f52b8074fe0 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 0x7f52b81a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 0x7f52b81a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57100/0 (socket says 172.21.15.64:57100) 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52b77fe700 1 -- 172.21.15.64:0/3837506935 learned_addr learned my addr 172.21.15.64:0/3837506935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.095+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8073a00 0x7f52b819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b67fc700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 0x7f52b819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b77fe700 1 -- 172.21.15.64:0/3837506935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 msgr2=0x7f52b819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b77fe700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 0x7f52b819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b77fe700 1 -- 172.21.15.64:0/3837506935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8073a00 msgr2=0x7f52b819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b77fe700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8073a00 0x7f52b819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b77fe700 1 -- 172.21.15.64:0/3837506935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0095f0 con 0x7f52b810d4c0 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b67fc700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 0x7f52b819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52b77fe700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 0x7f52b81a4bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f52a800e940 tx=0x7f52a800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f529ffff700 1 -- 172.21.15.64:0/3837506935 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80097b0 con 0x7f52b810d4c0 2024-05-08T22:06:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b0009c40 con 0x7f52b810d4c0 2024-05-08T22:06:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f529ffff700 1 -- 172.21.15.64:0/3837506935 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a800ee90 con 0x7f52b810d4c0 2024-05-08T22:06:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f529ffff700 1 -- 172.21.15.64:0/3837506935 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005520 con 0x7f52b810d4c0 2024-05-08T22:06:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.096+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b81a5af0 con 0x7f52b810d4c0 2024-05-08T22:06:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.101+0000 7f529ffff700 1 -- 172.21.15.64:0/3837506935 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f52a8004750 con 0x7f52b810d4c0 2024-05-08T22:06:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.101+0000 7f529ffff700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52a00a1120 0x7f52a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f529ffff700 1 -- 172.21.15.64:0/3837506935 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f52a8014070 con 0x7f52b810d4c0 2024-05-08T22:06:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 Processor -- start 2024-05-08T22:06:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52a00a1120 0x7f52a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 -- start start 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd926a80 0x5620bd926ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd927cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd928750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd87e0f0 con 0x5620bd926a80 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd87bab0 con 0x5620bd9273e0 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd8773a0 con 0x5620bd9282d0 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52b67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd927cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52b77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd928750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52b67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd927cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51828/0 (socket says 172.21.15.64:51828) 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52b67fc700 1 -- 172.21.15.64:0/709079694 learned_addr learned my addr 172.21.15.64:0/709079694 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.102+0000 7f52b77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd928750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:52466/0 (socket says 172.21.15.64:52466) 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52a00a1120 0x7f52a00a35e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f52ac005fd0 tx=0x7f52ac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd926a80 0x5620bd926ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b77fe700 1 -- 172.21.15.64:0/709079694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 msgr2=0x5620bd927cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b77fe700 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd927cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b77fe700 1 -- 172.21.15.64:0/709079694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd926a80 msgr2=0x5620bd926ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b77fe700 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd926a80 0x5620bd926ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b77fe700 1 -- 172.21.15.64:0/709079694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620bd6f8350 con 0x5620bd9282d0 2024-05-08T22:06:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f52b77fe700 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd928750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52a80ca660 tx=0x7f52a8099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e1ef17b8abe555 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f529d7fa700 1 -- 172.21.15.64:0/709079694 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80b1070 con 0x5620bd9282d0 2024-05-08T22:06:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f529d7fa700 1 -- 172.21.15.64:0/709079694 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a80a2430 con 0x5620bd9282d0 2024-05-08T22:06:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.103+0000 7f529d7fa700 1 -- 172.21.15.64:0/709079694 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a809a470 con 0x5620bd9282d0 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 -- 172.21.15.64:0/709079694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 msgr2=0x5620bd928750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd928750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52a80ca660 tx=0x7f52a8099c20 comp rx=0 tx=0).stop 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 -- 172.21.15.64:0/709079694 shutdown_connections 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd928750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd926a80 0x5620bd926ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/709079694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd927cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 -- 172.21.15.64:0/709079694 >> 172.21.15.64:0/709079694 conn(0x5620bd8851b0 msgr2=0x5620bd923440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 -- 172.21.15.64:0/709079694 shutdown_connections 2024-05-08T22:06:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 -- 172.21.15.64:0/709079694 wait complete. 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.104+0000 7f52c95fdb80 1 -- start start 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 Processor -- start 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd926a80 0x5620bd9308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd930de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 0x5620bd937e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd813500 con 0x5620bd9282d0 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd8131a0 con 0x5620bd9273e0 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd6fa640 con 0x5620bd926a80 2024-05-08T22:06:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52c95fdb80 10 client.0 _ll_register_callbacks cb 0x5620bd81cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 0x5620bd937e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 0x5620bd937e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57118/0 (socket says 172.21.15.64:57118) 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 -- 172.21.15.64:0/146782785 learned_addr learned my addr 172.21.15.64:0/146782785 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b67fc700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd930de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd926a80 0x5620bd9308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 -- 172.21.15.64:0/146782785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd926a80 msgr2=0x5620bd9308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd926a80 0x5620bd9308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 -- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 msgr2=0x5620bd930de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd930de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b77fe700 1 -- 172.21.15.64:0/146782785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b81a5af0 con 0x5620bd9282d0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.105+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd926a80 0x5620bd9308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f52b77fe700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 0x5620bd937e20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f52a8099f40 tx=0x7f52a800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414a90e407b3d77a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f529aff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f529aff5700 1 -- 172.21.15.64:0/146782785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80b1070 con 0x5620bd9282d0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f529aff5700 1 -- 172.21.15.64:0/146782785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f52a809a420 con 0x5620bd9282d0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620bd7d0210 con 0x5620bd9282d0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f529aff5700 1 -- 172.21.15.64:0/146782785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a80b82a0 con 0x5620bd9282d0 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f52c95fdb80 10 client.16038 fetch_fsmap learned FSMap version 45 2024-05-08T22:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.106+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5620bd6a4b00 con 0x5620bd9282d0 2024-05-08T22:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f529aff5700 1 -- 172.21.15.64:0/146782785 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620bd7d0210 con 0x5620bd9282d0 2024-05-08T22:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f529aff5700 1 -- 172.21.15.64:0/146782785 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f52a8095da0 con 0x5620bd9282d0 2024-05-08T22:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f52c95fdb80 10 client.16038 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f52c95fdb80 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f52c95fdb80 20 client.16038 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd94d2f0 0x5620bd94f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f52c95fdb80 4 client.16038 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.115+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620bd8f67b0 con 0x5620bd94d2f0 2024-05-08T22:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.121+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd94d2f0 0x5620bd94f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.142+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd94d2f0 0x5620bd94f6e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d8d7c4751b636d server_cookie=ba204a77d373bacc in_seq=0 out_seq=0 2024-05-08T22:06:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.144+0000 7f529aff5700 10 client.16038 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.154+0000 7f529aff5700 1 -- 172.21.15.64:0/146782785 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620bd8f67b0 con 0x5620bd94d2f0 2024-05-08T22:06:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.154+0000 7f529aff5700 10 client.16038 handle_command_reply: tid=0 2024-05-08T22:06:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.155+0000 7f52c95fdb80 1 client.16038 shutdown 2024-05-08T22:06:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.163+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 msgr2=0x5620bd937e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.163+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 0x5620bd937e20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f52a8099f40 tx=0x7f52a800cea0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 shutdown_connections 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd94d2f0 0x5620bd94f6e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9282d0 0x5620bd937e20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd926a80 0x5620bd9308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/146782785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd9273e0 0x5620bd930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 >> 172.21.15.64:0/146782785 conn(0x5620bd8851b0 msgr2=0x5620bd921860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.166+0000 7f529aff5700 0 client.16038 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.169+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 shutdown_connections 2024-05-08T22:06:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.179+0000 7f52c95fdb80 1 -- 172.21.15.64:0/146782785 wait complete. 2024-05-08T22:06:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.185+0000 7f52c95fdb80 20 client.16038 trim_cache size 0 max 16384 2024-05-08T22:06:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.212+0000 7f52c95fdb80 1 Processor -- start 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 -- start start 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd926e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd937de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd9494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd922c10 con 0x5620bd9273e0 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd930410 con 0x5620bd926a80 2024-05-08T22:06:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.213+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd92e520 con 0x5620bd9282d0 2024-05-08T22:06:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.215+0000 7f52b67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd937de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.215+0000 7f52b67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd937de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57122/0 (socket says 172.21.15.64:57122) 2024-05-08T22:06:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.215+0000 7f52b67fc700 1 -- 172.21.15.64:0/2577577825 learned_addr learned my addr 172.21.15.64:0/2577577825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.215+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd926e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.216+0000 7f52b67fc700 1 -- 172.21.15.64:0/2577577825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 msgr2=0x5620bd9494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.216+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd9494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.216+0000 7f52b67fc700 1 -- 172.21.15.64:0/2577577825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 msgr2=0x5620bd926e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.216+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd926e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.216+0000 7f52b67fc700 1 -- 172.21.15.64:0/2577577825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620bd83d500 con 0x5620bd9273e0 2024-05-08T22:06:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.216+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd937de0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f52b8074d00 tx=0x7f52b0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca06a3adcbe58da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.221+0000 7f529dffb700 1 -- 172.21.15.64:0/2577577825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000e040 con 0x5620bd9273e0 2024-05-08T22:06:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.221+0000 7f529dffb700 1 -- 172.21.15.64:0/2577577825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b0004a50 con 0x5620bd9273e0 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.221+0000 7f529dffb700 1 -- 172.21.15.64:0/2577577825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0005040 con 0x5620bd9273e0 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2577577825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 msgr2=0x5620bd937de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd937de0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f52b8074d00 tx=0x7f52b0009490 comp rx=0 tx=0).stop 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2577577825 shutdown_connections 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd926e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd9494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2577577825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd937de0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2577577825 >> 172.21.15.64:0/2577577825 conn(0x5620bd8851b0 msgr2=0x5620bd954030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2577577825 shutdown_connections 2024-05-08T22:06:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2577577825 wait complete. 2024-05-08T22:06:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.222+0000 7f52c95fdb80 1 -- start start 2024-05-08T22:06:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 Processor -- start 2024-05-08T22:06:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd9338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd933e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd936da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd9372e0 con 0x5620bd9273e0 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd937450 con 0x5620bd926a80 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd937610 con 0x5620bd9282d0 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52c95fdb80 10 client.0 _ll_register_callbacks cb 0x5620bd81cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52b6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd9338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52b6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd9338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51854/0 (socket says 172.21.15.64:51854) 2024-05-08T22:06:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.223+0000 7f52b6ffd700 1 -- 172.21.15.64:0/2297053141 learned_addr learned my addr 172.21.15.64:0/2297053141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.229+0000 7f52b77fe700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd936da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.230+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd933e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.231+0000 7f52b67fc700 1 -- 172.21.15.64:0/2297053141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 msgr2=0x5620bd936da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.231+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd936da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.231+0000 7f52b67fc700 1 -- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 msgr2=0x5620bd9338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.231+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd9338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.231+0000 7f52b67fc700 1 -- 172.21.15.64:0/2297053141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b8110260 con 0x5620bd9273e0 2024-05-08T22:06:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.232+0000 7f52b67fc700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd933e20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009790 tx=0x7f52b0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd20ccf5745faea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.232+0000 7f529aff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:06:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.232+0000 7f529aff5700 1 -- 172.21.15.64:0/2297053141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000e040 con 0x5620bd9273e0 2024-05-08T22:06:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f529aff5700 1 -- 172.21.15.64:0/2297053141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f52b0005250 con 0x5620bd9273e0 2024-05-08T22:06:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52a80a2430 con 0x5620bd9273e0 2024-05-08T22:06:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f529aff5700 1 -- 172.21.15.64:0/2297053141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b0005410 con 0x5620bd9273e0 2024-05-08T22:06:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f52c95fdb80 10 client.16044 fetch_fsmap learned FSMap version 45 2024-05-08T22:06:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5620bd6fcc70 con 0x5620bd9273e0 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f529aff5700 1 -- 172.21.15.64:0/2297053141 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a80a2430 con 0x5620bd9273e0 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.233+0000 7f529aff5700 1 -- 172.21.15.64:0/2297053141 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f52b000b050 con 0x5620bd9273e0 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52c95fdb80 10 client.16044 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52c95fdb80 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52c95fdb80 20 client.16044 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd9b5160 0x5620bd9b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52c95fdb80 4 client.16044 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620bd86fe60 con 0x5620bd9b5160 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd9b5160 0x5620bd9b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f529aff5700 10 client.16044 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.234+0000 7f52b6ffd700 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd9b5160 0x5620bd9b7580 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e57c39b6e77e0b2 server_cookie=82b1c0773ce0d900 in_seq=0 out_seq=0 2024-05-08T22:06:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.241+0000 7f529aff5700 1 -- 172.21.15.64:0/2297053141 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620bd86fe60 con 0x5620bd9b5160 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.241+0000 7f529aff5700 10 client.16044 handle_command_reply: tid=0 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 client.16044 shutdown 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 msgr2=0x5620bd933e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd933e20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009790 tx=0x7f52b0009920 comp rx=0 tx=0).stop 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 shutdown_connections 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620bd926a80 0x5620bd9338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5620bd9b5160 0x5620bd9b7580 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620bd9282d0 0x5620bd936da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 --2- 172.21.15.64:0/2297053141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620bd9273e0 0x5620bd933e20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 >> 172.21.15.64:0/2297053141 conn(0x5620bd8851b0 msgr2=0x5620bd923020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f529aff5700 0 client.16044 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 shutdown_connections 2024-05-08T22:06:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.242+0000 7f52c95fdb80 1 -- 172.21.15.64:0/2297053141 wait complete. 2024-05-08T22:06:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.243+0000 7f52c95fdb80 20 client.16044 trim_cache size 0 max 16384 2024-05-08T22:06:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:06:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:06:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:06:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52a00a1120 msgr2=0x7f52a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52a00a1120 0x7f52a00a35e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f52ac005fd0 tx=0x7f52ac009310 comp rx=0 tx=0).stop 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 msgr2=0x7f52b81a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 0x7f52b81a4bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f52a800e940 tx=0x7f52a800ec50 comp rx=0 tx=0).stop 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 shutdown_connections 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52a00a1120 0x7f52a00a35e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8073a00 0x7f52b819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b810d4c0 0x7f52b81a4bf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 --2- 172.21.15.64:0/3837506935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8074fe0 0x7f52b819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 >> 172.21.15.64:0/3837506935 conn(0x7f52b80fc460 msgr2=0x7f52b8100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 shutdown_connections 2024-05-08T22:06:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:21.245+0000 7f52bd443700 1 -- 172.21.15.64:0/3837506935 wait complete. 2024-05-08T22:06:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 690.0 s, 1569.3 tps, lat 63.978 ms stddev 80.883 2024-05-08T22:06:21.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:21 smithi186 ceph-mon[46619]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 753 op/s 2024-05-08T22:06:21.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:21 smithi064 ceph-mon[42500]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 753 op/s 2024-05-08T22:06:22.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:21 smithi160 ceph-mon[46962]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 753 op/s 2024-05-08T22:06:22.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:22 smithi186 ceph-mon[46619]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 646 op/s 2024-05-08T22:06:22.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:22 smithi064 ceph-mon[42500]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 646 op/s 2024-05-08T22:06:23.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:22 smithi160 ceph-mon[46962]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 646 op/s 2024-05-08T22:06:25.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:25 smithi186 ceph-mon[46619]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 611 op/s 2024-05-08T22:06:25.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:25 smithi064 ceph-mon[42500]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 611 op/s 2024-05-08T22:06:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:25 smithi160 ceph-mon[46962]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 611 op/s 2024-05-08T22:06:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 695.0 s, 1567.8 tps, lat 63.690 ms stddev 69.534 2024-05-08T22:06:26.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:26 smithi064 ceph-mon[42500]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 653 op/s 2024-05-08T22:06:27.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:26 smithi160 ceph-mon[46962]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 653 op/s 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:26 smithi186 ceph-mon[46619]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 653 op/s 2024-05-08T22:06:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:28 smithi064 ceph-mon[42500]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 623 op/s 2024-05-08T22:06:29.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:28 smithi160 ceph-mon[46962]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 623 op/s 2024-05-08T22:06:29.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:28 smithi186 ceph-mon[46619]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 623 op/s 2024-05-08T22:06:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 700.0 s, 1603.6 tps, lat 61.381 ms stddev 64.213 2024-05-08T22:06:31.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:31 smithi186 ceph-mon[46619]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 709 op/s 2024-05-08T22:06:31.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:31 smithi064 ceph-mon[42500]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 709 op/s 2024-05-08T22:06:32.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:31 smithi160 ceph-mon[46962]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 709 op/s 2024-05-08T22:06:33.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:33 smithi186 ceph-mon[46619]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.9 MiB/s wr, 633 op/s 2024-05-08T22:06:33.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:33 smithi064 ceph-mon[42500]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.9 MiB/s wr, 633 op/s 2024-05-08T22:06:34.052 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:33 smithi160 ceph-mon[46962]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.9 MiB/s wr, 633 op/s 2024-05-08T22:06:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/4230283498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 msgr2=0x7fa79c1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/4230283498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c1066a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7900098f0 tx=0x7fa790009c00 comp rx=0 tx=0).stop 2024-05-08T22:06:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/4230283498 shutdown_connections 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/4230283498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c106be0 0x7fa79c109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/4230283498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c101980 0x7fa79c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/4230283498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c1066a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/4230283498 >> 172.21.15.64:0/4230283498 conn(0x7fa79c0fb390 msgr2=0x7fa79c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/4230283498 shutdown_connections 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.387+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/4230283498 wait complete. 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 Processor -- start 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 -- start start 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 0x7fa79c194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 0x7fa79c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19c7c0 con 0x7fa79c106be0 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19c930 con 0x7fa79c1042b0 2024-05-08T22:06:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa7a1ec5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19caf0 con 0x7fa79c101980 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 0x7fa79c194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 0x7fa79c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 0x7fa79c194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:36374/0 (socket says 172.21.15.64:36374) 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 0x7fa79c19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50212/0 (socket says 172.21.15.64:50212) 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c195210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:34658/0 (socket says 172.21.15.64:34658) 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79bfff700 1 -- 172.21.15.64:0/988273934 learned_addr learned my addr 172.21.15.64:0/988273934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79b7fe700 1 -- 172.21.15.64:0/988273934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 msgr2=0x7fa79c195210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.388+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c195210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa79b7fe700 1 -- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 msgr2=0x7fa79c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 0x7fa79c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa79b7fe700 1 -- 172.21.15.64:0/988273934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0095f0 con 0x7fa79c101980 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa79bfff700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 0x7fa79c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 0x7fa79c194cd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00eb90 tx=0x7fa78c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa798ff9700 1 -- 172.21.15.64:0/988273934 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c009970 con 0x7fa79c101980 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa798ff9700 1 -- 172.21.15.64:0/988273934 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa78c0044e0 con 0x7fa79c101980 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.389+0000 7fa798ff9700 1 -- 172.21.15.64:0/988273934 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c003dd0 con 0x7fa79c101980 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.390+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa790009520 con 0x7fa79c101980 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.390+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c19d100 con 0x7fa79c101980 2024-05-08T22:06:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.392+0000 7fa798ff9700 1 -- 172.21.15.64:0/988273934 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa78c003f70 con 0x7fa79c101980 2024-05-08T22:06:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 Processor -- start 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 -- start start 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa798ff9700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7840a56b0 0x7fa7840a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 0x5578f01d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d22d0 0x5578f01d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01280f0 con 0x5578f01d0a80 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f0125ab0 con 0x5578f01d13e0 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.394+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01213a0 con 0x5578f01d22d0 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79bfff700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7840a56b0 0x7fa7840a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa798ff9700 1 -- 172.21.15.64:0/988273934 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa78c014070 con 0x7fa79c101980 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 0x5578f01d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 0x5578f01d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50228/0 (socket says 172.21.15.64:50228) 2024-05-08T22:06:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 -- 172.21.15.64:0/1920584958 learned_addr learned my addr 172.21.15.64:0/1920584958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79bfff700 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d22d0 0x5578f01d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79affd700 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 -- 172.21.15.64:0/1920584958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d22d0 msgr2=0x5578f01d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d22d0 0x5578f01d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 -- 172.21.15.64:0/1920584958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 msgr2=0x5578f01d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 -- 172.21.15.64:0/1920584958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578effa2350 con 0x5578f01d0a80 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79affd700 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79bfff700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7840a56b0 0x7fa7840a7b70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa794009550 tx=0x7fa794007c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 0x5578f01d0ea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa78c000c00 tx=0x7fa78c099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05e32a6af29a052 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa781ffb700 1 -- 172.21.15.64:0/1920584958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00fbb0 con 0x5578f01d0a80 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa781ffb700 1 -- 172.21.15.64:0/1920584958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa78c0177e0 con 0x5578f01d0a80 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.395+0000 7fa781ffb700 1 -- 172.21.15.64:0/1920584958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0981e0 con 0x5578f01d0a80 2024-05-08T22:06:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1920584958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 msgr2=0x5578f01d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 0x5578f01d0ea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa78c000c00 tx=0x7fa78c099be0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1920584958 shutdown_connections 2024-05-08T22:06:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d22d0 0x5578f01d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d0a80 0x5578f01d0ea0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1920584958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.396+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1920584958 >> 172.21.15.64:0/1920584958 conn(0x5578f012f1b0 msgr2=0x5578f01cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.397+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1920584958 shutdown_connections 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.398+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1920584958 wait complete. 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 -- start start 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 Processor -- start 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 0x5578f01da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f00bd500 con 0x5578f01d22d0 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f00bd1a0 con 0x5578f01d0a80 2024-05-08T22:06:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578effa4640 con 0x5578f01d13e0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa7ae07fb80 10 client.0 _ll_register_callbacks cb 0x5578f00c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa79affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa79affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:36400/0 (socket says 172.21.15.64:36400) 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.399+0000 7fa79affd700 1 -- 172.21.15.64:0/2064493450 learned_addr learned my addr 172.21.15.64:0/2064493450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa79affd700 1 -- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 msgr2=0x5578f01da8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa79affd700 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 0x5578f01da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa79affd700 1 -- 172.21.15.64:0/2064493450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 msgr2=0x5578f01e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa79affd700 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa79affd700 1 -- 172.21.15.64:0/2064493450 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c19d100 con 0x5578f01d13e0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa79affd700 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dadf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa79000fa40 tx=0x7fa79000f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fa08e774d51bdbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa77affd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:06:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa77affd700 1 -- 172.21.15.64:0/2064493450 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790025070 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa77affd700 1 -- 172.21.15.64:0/2064493450 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa790005040 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578f007a210 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa77affd700 1 -- 172.21.15.64:0/2064493450 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa79001c070 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa7ae07fb80 10 client.25690 fetch_fsmap learned FSMap version 45 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5578eff4eb00 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.400+0000 7fa77affd700 1 -- 172.21.15.64:0/2064493450 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578f007a210 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa77affd700 1 -- 172.21.15.64:0/2064493450 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fa790028030 con 0x5578f01d13e0 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa7ae07fb80 10 client.25690 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa7ae07fb80 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa7ae07fb80 20 client.25690 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f01e6730 0x5578f01e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa7ae07fb80 4 client.25690 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578f01a07b0 con 0x5578f01e6730 2024-05-08T22:06:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f01e6730 0x5578f01e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa77affd700 10 client.25690 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.401+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f01e6730 0x5578f01e8b50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b878150b457dc23c server_cookie=e050d0c9b2372ef9 in_seq=0 out_seq=0 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.402+0000 7fa77affd700 1 -- 172.21.15.64:0/2064493450 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578f01a07b0 con 0x5578f01e6730 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.402+0000 7fa77affd700 10 client.25690 handle_command_reply: tid=0 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.402+0000 7fa7ae07fb80 1 client.25690 shutdown 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 msgr2=0x5578f01dadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dadf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa79000fa40 tx=0x7fa79000f960 comp rx=0 tx=0).stop 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 shutdown_connections 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 0x5578f01da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f01e6730 0x5578f01e8b50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2064493450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dadf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 >> 172.21.15.64:0/2064493450 conn(0x5578f012f1b0 msgr2=0x5578f01cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa77affd700 0 client.25690 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 shutdown_connections 2024-05-08T22:06:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2064493450 wait complete. 2024-05-08T22:06:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.403+0000 7fa7ae07fb80 20 client.25690 trim_cache size 0 max 16384 2024-05-08T22:06:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 Processor -- start 2024-05-08T22:06:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 -- start start 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d0a80 0x5578f01e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01dbaf0 con 0x5578f01d22d0 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01edd60 con 0x5578f01d13e0 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.441+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01d05e0 con 0x5578f01d0a80 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:34678/0 (socket says 172.21.15.64:34678) 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 -- 172.21.15.64:0/2361447740 learned_addr learned my addr 172.21.15.64:0/2361447740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 -- 172.21.15.64:0/2361447740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d0a80 msgr2=0x5578f01e8b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:06:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d0a80 0x5578f01e8b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 -- 172.21.15.64:0/2361447740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 msgr2=0x5578f01e24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.442+0000 7fa79affd700 1 -- 172.21.15.64:0/2361447740 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578f00e7500 con 0x5578f01d13e0 2024-05-08T22:06:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.458+0000 7fa79affd700 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d0e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa790005a00 tx=0x7fa790009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648c040d368d8223 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.469+0000 7fa7827fc700 1 -- 172.21.15.64:0/2361447740 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790025030 con 0x5578f01d13e0 2024-05-08T22:06:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.469+0000 7fa7827fc700 1 -- 172.21.15.64:0/2361447740 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79001d040 con 0x5578f01d13e0 2024-05-08T22:06:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.469+0000 7fa7827fc700 1 -- 172.21.15.64:0/2361447740 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79001c040 con 0x5578f01d13e0 2024-05-08T22:06:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.471+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2361447740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 msgr2=0x5578f01d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.471+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d0e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa790005a00 tx=0x7fa790009c00 comp rx=0 tx=0).stop 2024-05-08T22:06:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.472+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2361447740 shutdown_connections 2024-05-08T22:06:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.472+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.472+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d0a80 0x5578f01e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.472+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/2361447740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d13e0 0x5578f01d0e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.472+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2361447740 >> 172.21.15.64:0/2361447740 conn(0x5578f012f1b0 msgr2=0x5578f01d8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.485+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2361447740 shutdown_connections 2024-05-08T22:06:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.487+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/2361447740 wait complete. 2024-05-08T22:06:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.487+0000 7fa7ae07fb80 1 -- start start 2024-05-08T22:06:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.487+0000 7fa7ae07fb80 1 Processor -- start 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.487+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 0x5578f01dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa7ae07fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01e0c20 con 0x5578f01d22d0 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01e0d90 con 0x5578f01d0a80 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa7ae07fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01e0f50 con 0x5578f01d13e0 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa7ae07fb80 10 client.0 _ll_register_callbacks cb 0x5578f00c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50258/0 (socket says 172.21.15.64:50258) 2024-05-08T22:06:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 -- 172.21.15.64:0/1971867610 learned_addr learned my addr 172.21.15.64:0/1971867610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 -- 172.21.15.64:0/1971867610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 msgr2=0x5578f01dd9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:06:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dd9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 -- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 msgr2=0x5578f01dd3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:06:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 0x5578f01dd3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 -- 172.21.15.64:0/1971867610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578eff4eb00 con 0x5578f01d22d0 2024-05-08T22:06:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa79bfff700 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e06e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa794009c10 tx=0x7fa794004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75a25fa4181453d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.488+0000 7fa77affd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:06:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.489+0000 7fa77affd700 1 -- 172.21.15.64:0/1971867610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794024800 con 0x5578f01d22d0 2024-05-08T22:06:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.489+0000 7fa77affd700 1 -- 172.21.15.64:0/1971867610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa794024960 con 0x5578f01d22d0 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.489+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa78c0177e0 con 0x5578f01d22d0 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.489+0000 7fa77affd700 1 -- 172.21.15.64:0/1971867610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa794024b20 con 0x5578f01d22d0 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.489+0000 7fa7ae07fb80 10 client.16059 fetch_fsmap learned FSMap version 45 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.489+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5578effa6c70 con 0x5578f01d22d0 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa77affd700 1 -- 172.21.15.64:0/1971867610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa78c0177e0 con 0x5578f01d22d0 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa77affd700 1 -- 172.21.15.64:0/1971867610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fa79402c3e0 con 0x5578f01d22d0 2024-05-08T22:06:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa7ae07fb80 10 client.16059 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:06:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa7ae07fb80 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa7ae07fb80 20 client.16059 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f02569d0 0x5578f0258df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa7ae07fb80 4 client.16059 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:06:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.490+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578f0119e60 con 0x5578f02569d0 2024-05-08T22:06:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.491+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f02569d0 0x5578f0258df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.498+0000 7fa77affd700 10 client.16059 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.498+0000 7fa79b7fe700 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f02569d0 0x5578f0258df0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356788b6eae3e3fa server_cookie=7e43928313836e46 in_seq=0 out_seq=0 2024-05-08T22:06:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa77affd700 1 -- 172.21.15.64:0/1971867610 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578f0119e60 con 0x5578f02569d0 2024-05-08T22:06:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa77affd700 10 client.16059 handle_command_reply: tid=0 2024-05-08T22:06:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 client.16059 shutdown 2024-05-08T22:06:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 msgr2=0x5578f01e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e06e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa794009c10 tx=0x7fa794004ad0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 shutdown_connections 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5578f01d0a80 0x5578f01dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5578f02569d0 0x5578f0258df0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f01d22d0 0x5578f01e06e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 --2- 172.21.15.64:0/1971867610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578f01d13e0 0x5578f01dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 >> 172.21.15.64:0/1971867610 conn(0x5578f012f1b0 msgr2=0x5578f01d87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.515+0000 7fa77affd700 0 client.16059 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 shutdown_connections 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7ae07fb80 1 -- 172.21.15.64:0/1971867610 wait complete. 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7ae07fb80 20 client.16059 trim_cache size 0 max 16384 2024-05-08T22:06:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:06:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:06:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:06:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:06:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7840a56b0 msgr2=0x7fa7840a7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7840a56b0 0x7fa7840a7b70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa794009550 tx=0x7fa794007c20 comp rx=0 tx=0).stop 2024-05-08T22:06:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 msgr2=0x7fa79c194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 0x7fa79c194cd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00eb90 tx=0x7fa78c00eea0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 shutdown_connections 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa7840a56b0 0x7fa7840a7b70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c106be0 0x7fa79c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa79c101980 0x7fa79c194cd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 --2- 172.21.15.64:0/988273934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa79c1042b0 0x7fa79c195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.516+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 >> 172.21.15.64:0/988273934 conn(0x7fa79c0fb390 msgr2=0x7fa79c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.517+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 shutdown_connections 2024-05-08T22:06:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:34.517+0000 7fa7a1ec5700 1 -- 172.21.15.64:0/988273934 wait complete. 2024-05-08T22:06:35.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:35 smithi186 ceph-mon[46619]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.5 MiB/s wr, 651 op/s 2024-05-08T22:06:35.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:35 smithi064 ceph-mon[42500]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.5 MiB/s wr, 651 op/s 2024-05-08T22:06:36.050 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:35 smithi160 ceph-mon[46962]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.5 MiB/s wr, 651 op/s 2024-05-08T22:06:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 705.0 s, 1487.6 tps, lat 67.699 ms stddev 71.996 2024-05-08T22:06:37.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:37 smithi186 ceph-mon[46619]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 715 op/s 2024-05-08T22:06:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:37 smithi064 ceph-mon[42500]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 715 op/s 2024-05-08T22:06:38.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:37 smithi160 ceph-mon[46962]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 715 op/s 2024-05-08T22:06:39.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:39 smithi186 ceph-mon[46619]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.1 MiB/s wr, 659 op/s 2024-05-08T22:06:39.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:06:39.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:06:39.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:06:39.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:39 smithi064 ceph-mon[42500]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.1 MiB/s wr, 659 op/s 2024-05-08T22:06:39.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:06:39.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:06:39.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:06:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:39 smithi160 ceph-mon[46962]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.1 MiB/s wr, 659 op/s 2024-05-08T22:06:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:06:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:06:40.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:06:40.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:06:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 710.0 s, 1748.1 tps, lat 58.038 ms stddev 66.915 2024-05-08T22:06:41.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:41 smithi186 ceph-mon[46619]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 MiB/s wr, 752 op/s 2024-05-08T22:06:41.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:41 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:06:41.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:41 smithi064 ceph-mon[42500]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 MiB/s wr, 752 op/s 2024-05-08T22:06:41.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:41 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:06:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:41 smithi160 ceph-mon[46962]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 MiB/s wr, 752 op/s 2024-05-08T22:06:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:41 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:06:43.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:43 smithi186 ceph-mon[46619]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 MiB/s wr, 717 op/s 2024-05-08T22:06:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:43 smithi064 ceph-mon[42500]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 MiB/s wr, 717 op/s 2024-05-08T22:06:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:43 smithi160 ceph-mon[46962]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 MiB/s wr, 717 op/s 2024-05-08T22:06:45.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:45 smithi186 ceph-mon[46619]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.8 MiB/s wr, 697 op/s 2024-05-08T22:06:45.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:45 smithi064 ceph-mon[42500]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.8 MiB/s wr, 697 op/s 2024-05-08T22:06:46.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:45 smithi160 ceph-mon[46962]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.8 MiB/s wr, 697 op/s 2024-05-08T22:06:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 715.0 s, 1462.8 tps, lat 65.878 ms stddev 78.544 2024-05-08T22:06:46.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:46 smithi186 ceph-mon[46619]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 739 op/s 2024-05-08T22:06:46.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:46 smithi064 ceph-mon[42500]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 739 op/s 2024-05-08T22:06:47.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:46 smithi160 ceph-mon[46962]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 739 op/s 2024-05-08T22:06:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 -- 172.21.15.64:0/2210168728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8101110 msgr2=0x7f29a81079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 --2- 172.21.15.64:0/2210168728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8101110 0x7f29a81079c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f29a000b0a0 tx=0x7f29a000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 -- 172.21.15.64:0/2210168728 shutdown_connections 2024-05-08T22:06:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 --2- 172.21.15.64:0/2210168728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8101110 0x7f29a81079c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 --2- 172.21.15.64:0/2210168728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29a80ff580 0x7f29a80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 --2- 172.21.15.64:0/2210168728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8100770 0x7f29a8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 -- 172.21.15.64:0/2210168728 >> 172.21.15.64:0/2210168728 conn(0x7f29a80faaf0 msgr2=0x7f29a80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 -- 172.21.15.64:0/2210168728 shutdown_connections 2024-05-08T22:06:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.698+0000 7f29acba0700 1 -- 172.21.15.64:0/2210168728 wait complete. 2024-05-08T22:06:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 Processor -- start 2024-05-08T22:06:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 -- start start 2024-05-08T22:06:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 0x7f29a8197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8100770 0x7f29a8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29a8101110 0x7f29a819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a819eb40 con 0x7f29a8101110 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a819ec80 con 0x7f29a8100770 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29acba0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a819ee10 con 0x7f29a80ff580 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29a659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 0x7f29a8197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29a659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 0x7f29a8197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:52994/0 (socket says 172.21.15.64:52994) 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.699+0000 7f29a659c700 1 -- 172.21.15.64:0/627483604 learned_addr learned my addr 172.21.15.64:0/627483604 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29a659c700 1 -- 172.21.15.64:0/627483604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8100770 msgr2=0x7f29a8197550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29a659c700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8100770 0x7f29a8197550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29a659c700 1 -- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29a8101110 msgr2=0x7f29a819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29a659c700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29a8101110 0x7f29a819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29a659c700 1 -- 172.21.15.64:0/627483604 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009d00 con 0x7f29a80ff580 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29a659c700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 0x7f29a8197010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f299800b500 tx=0x7f299800b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29977fe700 1 -- 172.21.15.64:0/627483604 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998011830 con 0x7f29a80ff580 2024-05-08T22:06:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29977fe700 1 -- 172.21.15.64:0/627483604 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2998011e50 con 0x7f29a80ff580 2024-05-08T22:06:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.700+0000 7f29977fe700 1 -- 172.21.15.64:0/627483604 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800f2e0 con 0x7f29a80ff580 2024-05-08T22:06:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.701+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a819f0f0 con 0x7f29a80ff580 2024-05-08T22:06:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.701+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a819f5b0 con 0x7f29a80ff580 2024-05-08T22:06:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.704+0000 7f29977fe700 1 -- 172.21.15.64:0/627483604 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2998011990 con 0x7f29a80ff580 2024-05-08T22:06:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.704+0000 7f29977fe700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f29900a1220 0x7f29900a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.705+0000 7f29977fe700 1 -- 172.21.15.64:0/627483604 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f29980cc3d0 con 0x7f29a80ff580 2024-05-08T22:06:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.705+0000 7f29b8d5ab80 1 Processor -- start 2024-05-08T22:06:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.709+0000 7f29b8d5ab80 1 -- start start 2024-05-08T22:06:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 0x561169972dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a5d9b700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f29900a1220 0x7f29900a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169973c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x561169974680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611698ca0f0 con 0x561169974200 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611698c7ab0 con 0x5611699729b0 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611698c33a0 con 0x561169973310 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 0x561169972dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 0x561169972dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:42872/0 (socket says 172.21.15.64:42872) 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 -- 172.21.15.64:0/2038511708 learned_addr learned my addr 172.21.15.64:0/2038511708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a5d9b700 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169973c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 -- 172.21.15.64:0/2038511708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 msgr2=0x561169973c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169973c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 -- 172.21.15.64:0/2038511708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 msgr2=0x561169974680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x561169974680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 -- 172.21.15.64:0/2038511708 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561169744350 con 0x5611699729b0 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.710+0000 7f29a659c700 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 0x561169972dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2998099760 tx=0x7f2998004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecf001494e49c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.711+0000 7f2994ff9700 1 -- 172.21.15.64:0/2038511708 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998099ed0 con 0x5611699729b0 2024-05-08T22:06:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.711+0000 7f2994ff9700 1 -- 172.21.15.64:0/2038511708 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29980997d0 con 0x5611699729b0 2024-05-08T22:06:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.711+0000 7f2994ff9700 1 -- 172.21.15.64:0/2038511708 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998097c90 con 0x5611699729b0 2024-05-08T22:06:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2038511708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 msgr2=0x561169972dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 0x561169972dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2998099760 tx=0x7f2998004750 comp rx=0 tx=0).stop 2024-05-08T22:06:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2038511708 shutdown_connections 2024-05-08T22:06:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611699729b0 0x561169972dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x561169974680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2038511708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169973c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2038511708 >> 172.21.15.64:0/2038511708 conn(0x5611698d11b0 msgr2=0x56116996f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.712+0000 7f29a5d9b700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f29900a1220 0x7f29900a36e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f299c009f20 tx=0x7f299c0126d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.713+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2038511708 shutdown_connections 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.713+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2038511708 wait complete. 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.713+0000 7f29b8d5ab80 1 -- start start 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.713+0000 7f29b8d5ab80 1 Processor -- start 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116997c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x56116997ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 0x561169983ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116985f500 con 0x5611699729b0 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116985f1a0 con 0x561169973310 2024-05-08T22:06:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561169746640 con 0x561169974200 2024-05-08T22:06:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29b8d5ab80 10 client.0 _ll_register_callbacks cb 0x561169868ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29a6d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 0x561169983ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29a6d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 0x561169983ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53008/0 (socket says 172.21.15.64:53008) 2024-05-08T22:06:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29a6d9d700 1 -- 172.21.15.64:0/3988966722 learned_addr learned my addr 172.21.15.64:0/3988966722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.714+0000 7f29a659c700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116997c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a5d9b700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x56116997ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a6d9d700 1 -- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 msgr2=0x56116997ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a6d9d700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x56116997ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a6d9d700 1 -- 172.21.15.64:0/3988966722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 msgr2=0x56116997c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a6d9d700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116997c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a6d9d700 1 -- 172.21.15.64:0/3988966722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a819f5b0 con 0x561169974200 2024-05-08T22:06:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f29a6d9d700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 0x561169983ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009850 tx=0x7f29a0008ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592cec2043d244f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f298dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f298dffb700 1 -- 172.21.15.64:0/3988966722 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000e040 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.715+0000 7f298dffb700 1 -- 172.21.15.64:0/3988966722 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f29a000bab0 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.716+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56116981c210 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.716+0000 7f298dffb700 1 -- 172.21.15.64:0/3988966722 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a000bc90 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.716+0000 7f29b8d5ab80 10 client.25702 fetch_fsmap learned FSMap version 46 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.716+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5611696f0b00 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.716+0000 7f298dffb700 1 -- 172.21.15.64:0/3988966722 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56116981c210 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f298dffb700 1 -- 172.21.15.64:0/3988966722 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f29a0019040 con 0x561169974200 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29b8d5ab80 10 client.25702 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29b8d5ab80 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29b8d5ab80 20 client.25702 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169999790 0x56116999bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29b8d5ab80 4 client.25702 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611699427b0 con 0x561169999790 2024-05-08T22:06:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29a659c700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169999790 0x56116999bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.717+0000 7f29a659c700 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169999790 0x56116999bbb0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e865d58f4c72de server_cookie=4aeb1894f58aad84 in_seq=0 out_seq=0 2024-05-08T22:06:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.718+0000 7f298dffb700 10 client.25702 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.718+0000 7f298dffb700 1 -- 172.21.15.64:0/3988966722 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611699427b0 con 0x561169999790 2024-05-08T22:06:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.718+0000 7f298dffb700 10 client.25702 handle_command_reply: tid=0 2024-05-08T22:06:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 client.25702 shutdown 2024-05-08T22:06:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 msgr2=0x561169983ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 0x561169983ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009850 tx=0x7f29a0008ee0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 shutdown_connections 2024-05-08T22:06:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169999790 0x56116999bbb0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169974200 0x561169983ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116997c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/3988966722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x56116997ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 >> 172.21.15.64:0/3988966722 conn(0x5611698d11b0 msgr2=0x56116996d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.722+0000 7f298dffb700 0 client.25702 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.723+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 shutdown_connections 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.723+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/3988966722 wait complete. 2024-05-08T22:06:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.723+0000 7f29b8d5ab80 20 client.25702 trim_cache size 0 max 16384 2024-05-08T22:06:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.759+0000 7f29b8d5ab80 1 Processor -- start 2024-05-08T22:06:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 -- start start 2024-05-08T22:06:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116996f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169983e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169974200 0x56116996f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611699725c0 con 0x5611699729b0 2024-05-08T22:06:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116996ec10 con 0x561169974200 2024-05-08T22:06:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.766+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116997c4c0 con 0x561169973310 2024-05-08T22:06:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116996f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116996f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39028/0 (socket says 172.21.15.64:39028) 2024-05-08T22:06:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 -- 172.21.15.64:0/4059639581 learned_addr learned my addr 172.21.15.64:0/4059639581 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a6d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169974200 0x56116996f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 -- 172.21.15.64:0/4059639581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 msgr2=0x561169983e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:06:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169983e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 -- 172.21.15.64:0/4059639581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169974200 msgr2=0x56116996f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169974200 0x56116996f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.770+0000 7f29a659c700 1 -- 172.21.15.64:0/4059639581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561169889500 con 0x5611699729b0 2024-05-08T22:06:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.772+0000 7f29a659c700 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116996f020 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2998095cd0 tx=0x7f29980109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020187355ccedbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.777+0000 7f29957fa700 1 -- 172.21.15.64:0/4059639581 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980b7070 con 0x5611699729b0 2024-05-08T22:06:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.777+0000 7f29957fa700 1 -- 172.21.15.64:0/4059639581 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f299809d600 con 0x5611699729b0 2024-05-08T22:06:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.777+0000 7f29957fa700 1 -- 172.21.15.64:0/4059639581 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998099a30 con 0x5611699729b0 2024-05-08T22:06:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.778+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/4059639581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 msgr2=0x56116996f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.778+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116996f020 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2998095cd0 tx=0x7f29980109c0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.782+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/4059639581 shutdown_connections 2024-05-08T22:06:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.782+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169974200 0x56116996f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.782+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611699729b0 0x56116996f020 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.782+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/4059639581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561169973310 0x561169983e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.782+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/4059639581 >> 172.21.15.64:0/4059639581 conn(0x5611698d11b0 msgr2=0x56116997c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.784+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/4059639581 shutdown_connections 2024-05-08T22:06:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.784+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/4059639581 wait complete. 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.784+0000 7f29b8d5ab80 1 -- start start 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.784+0000 7f29b8d5ab80 1 Processor -- start 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611699729b0 0x5611699813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x561169981900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x5611699828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561169982e00 con 0x561169974200 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561169982f70 con 0x561169973310 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611699830e0 con 0x5611699729b0 2024-05-08T22:06:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29b8d5ab80 10 client.0 _ll_register_callbacks cb 0x561169868ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x5611699828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x5611699828c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39042/0 (socket says 172.21.15.64:39042) 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 -- 172.21.15.64:0/2857009518 learned_addr learned my addr 172.21.15.64:0/2857009518 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 -- 172.21.15.64:0/2857009518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611699729b0 msgr2=0x5611699813c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611699729b0 0x5611699813c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 -- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 msgr2=0x561169981900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x561169981900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.785+0000 7f29a6d9d700 1 -- 172.21.15.64:0/2857009518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561169744350 con 0x561169974200 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f29a6d9d700 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x5611699828c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f29a0002230 tx=0x7f29a001d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d45c0b427c777f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f298dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f298dffb700 1 -- 172.21.15.64:0/2857009518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000e070 con 0x561169974200 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f298dffb700 1 -- 172.21.15.64:0/2857009518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f29a00060e0 con 0x561169974200 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29980997d0 con 0x561169974200 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f298dffb700 1 -- 172.21.15.64:0/2857009518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a00077f0 con 0x561169974200 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f29b8d5ab80 10 client.16068 fetch_fsmap learned FSMap version 46 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.786+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561169748c70 con 0x561169974200 2024-05-08T22:06:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f298dffb700 1 -- 172.21.15.64:0/2857009518 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a0035030 con 0x561169974200 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f298dffb700 1 -- 172.21.15.64:0/2857009518 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f29a0019070 con 0x561169974200 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29b8d5ab80 10 client.16068 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29b8d5ab80 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29b8d5ab80 20 client.16068 populate_metadata read hostname 'smithi064' 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169a01130 0x561169a03550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29b8d5ab80 4 client.16068 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611698bbe60 con 0x561169a01130 2024-05-08T22:06:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.787+0000 7f29a659c700 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169a01130 0x561169a03550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.788+0000 7f298dffb700 10 client.16068 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.788+0000 7f29a659c700 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169a01130 0x561169a03550 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e26555b74c4382e server_cookie=625a1fb58c48b818 in_seq=0 out_seq=0 2024-05-08T22:06:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.804+0000 7f298dffb700 1 -- 172.21.15.64:0/2857009518 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611698bbe60 con 0x561169a01130 2024-05-08T22:06:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.804+0000 7f298dffb700 10 client.16068 handle_command_reply: tid=0 2024-05-08T22:06:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.805+0000 7f29b8d5ab80 1 client.16068 shutdown 2024-05-08T22:06:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.805+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 msgr2=0x5611699828c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.805+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x5611699828c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f29a0002230 tx=0x7f29a001d720 comp rx=0 tx=0).stop 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 shutdown_connections 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561169a01130 0x561169a03550 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561169974200 0x5611699828c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611699729b0 0x5611699813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f298dffb700 0 client.16068 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 --2- 172.21.15.64:0/2857009518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561169973310 0x561169981900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 >> 172.21.15.64:0/2857009518 conn(0x5611698d11b0 msgr2=0x56116997c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 shutdown_connections 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 1 -- 172.21.15.64:0/2857009518 wait complete. 2024-05-08T22:06:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.806+0000 7f29b8d5ab80 20 client.16068 trim_cache size 0 max 16384 2024-05-08T22:06:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:06:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:06:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:06:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f29900a1220 msgr2=0x7f29900a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f29900a1220 0x7f29900a36e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f299c009f20 tx=0x7f299c0126d0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 msgr2=0x7f29a8197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 0x7f29a8197010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f299800b500 tx=0x7f299800b8c0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 shutdown_connections 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f29900a1220 0x7f29900a36e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29a8101110 0x7f29a819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a80ff580 0x7f29a8197010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 --2- 172.21.15.64:0/627483604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a8100770 0x7f29a8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 >> 172.21.15.64:0/627483604 conn(0x7f29a80faaf0 msgr2=0x7f29a8103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 shutdown_connections 2024-05-08T22:06:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:06:47.807+0000 7f29acba0700 1 -- 172.21.15.64:0/627483604 wait complete. 2024-05-08T22:06:49.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:49 smithi186 ceph-mon[46619]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.6 MiB/s wr, 675 op/s 2024-05-08T22:06:49.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:49 smithi064 ceph-mon[42500]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.6 MiB/s wr, 675 op/s 2024-05-08T22:06:50.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:49 smithi160 ceph-mon[46962]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.6 MiB/s wr, 675 op/s 2024-05-08T22:06:50.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:50 smithi064 ceph-mon[42500]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 861 op/s 2024-05-08T22:06:51.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:50 smithi160 ceph-mon[46962]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 861 op/s 2024-05-08T22:06:51.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:50 smithi186 ceph-mon[46619]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 861 op/s 2024-05-08T22:06:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 720.0 s, 1190.4 tps, lat 86.230 ms stddev 101.025 2024-05-08T22:06:53.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:53 smithi186 ceph-mon[46619]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 791 op/s 2024-05-08T22:06:53.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:53 smithi064 ceph-mon[42500]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 791 op/s 2024-05-08T22:06:54.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:53 smithi160 ceph-mon[46962]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 791 op/s 2024-05-08T22:06:55.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:55 smithi186 ceph-mon[46619]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 729 op/s 2024-05-08T22:06:55.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:55 smithi064 ceph-mon[42500]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 729 op/s 2024-05-08T22:06:56.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:55 smithi160 ceph-mon[46962]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 729 op/s 2024-05-08T22:06:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 725.0 s, 1393.7 tps, lat 71.485 ms stddev 76.428 2024-05-08T22:06:57.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:57 smithi186 ceph-mon[46619]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 778 op/s 2024-05-08T22:06:57.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:57 smithi064 ceph-mon[42500]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 778 op/s 2024-05-08T22:06:58.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:57 smithi160 ceph-mon[46962]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 778 op/s 2024-05-08T22:06:59.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:06:59 smithi186 ceph-mon[46619]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 721 op/s 2024-05-08T22:06:59.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:06:59 smithi064 ceph-mon[42500]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 721 op/s 2024-05-08T22:07:00.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:06:59 smithi160 ceph-mon[46962]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 721 op/s 2024-05-08T22:07:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.993+0000 7f86d6208700 1 -- 172.21.15.64:0/472795000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 msgr2=0x7f86d0104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.993+0000 7f86d6208700 1 --2- 172.21.15.64:0/472795000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d0104790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f86c00099c0 tx=0x7f86c0009cd0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.994+0000 7f86d6208700 1 -- 172.21.15.64:0/472795000 shutdown_connections 2024-05-08T22:07:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.994+0000 7f86d6208700 1 --2- 172.21.15.64:0/472795000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d0104cd0 0x7f86d010b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.994+0000 7f86d6208700 1 --2- 172.21.15.64:0/472795000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86d0103140 0x7f86d0103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.994+0000 7f86d6208700 1 --2- 172.21.15.64:0/472795000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d0104790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:00.994+0000 7f86d6208700 1 -- 172.21.15.64:0/472795000 >> 172.21.15.64:0/472795000 conn(0x7f86d00fe6b0 msgr2=0x7f86d0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.003+0000 7f86d6208700 1 -- 172.21.15.64:0/472795000 shutdown_connections 2024-05-08T22:07:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.004+0000 7f86d6208700 1 -- 172.21.15.64:0/472795000 wait complete. 2024-05-08T22:07:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.004+0000 7f86d6208700 1 Processor -- start 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 -- start start 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d0103140 0x7f86d0194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d01952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86d0104cd0 0x7f86d019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019c8a0 con 0x7f86d0103140 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01adac0 con 0x7f86d0104330 2024-05-08T22:07:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.007+0000 7f86d6208700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01adc00 con 0x7f86d0104cd0 2024-05-08T22:07:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d01952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d01952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:46396/0 (socket says 172.21.15.64:46396) 2024-05-08T22:07:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 -- 172.21.15.64:0/38576283 learned_addr learned my addr 172.21.15.64:0/38576283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 -- 172.21.15.64:0/38576283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86d0104cd0 msgr2=0x7f86d019c2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86d0104cd0 0x7f86d019c2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 -- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d0103140 msgr2=0x7f86d0194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d0103140 0x7f86d0194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.008+0000 7f86ceffd700 1 -- 172.21.15.64:0/38576283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00095f0 con 0x7f86d0104330 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.009+0000 7f86ceffd700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d01952b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005f50 tx=0x7f86c0003ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.009+0000 7f86ccff9700 1 -- 172.21.15.64:0/38576283 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c001d070 con 0x7f86d0104330 2024-05-08T22:07:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.009+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d01ade30 con 0x7f86d0104330 2024-05-08T22:07:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.009+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01ae250 con 0x7f86d0104330 2024-05-08T22:07:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.010+0000 7f86ccff9700 1 -- 172.21.15.64:0/38576283 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c0005580 con 0x7f86d0104330 2024-05-08T22:07:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.010+0000 7f86ccff9700 1 -- 172.21.15.64:0/38576283 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000fd60 con 0x7f86d0104330 2024-05-08T22:07:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.014+0000 7f86e23c2b80 1 Processor -- start 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 -- start start 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94caa80 0x5631e94caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 0x5631e94cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94220f0 con 0x5631e94cb3e0 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e941fab0 con 0x5631e94cc2d0 2024-05-08T22:07:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.017+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e941b3a0 con 0x5631e94caa80 2024-05-08T22:07:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.018+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 0x5631e94cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.018+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 0x5631e94cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60700/0 (socket says 172.21.15.64:60700) 2024-05-08T22:07:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.018+0000 7f86cf7fe700 1 -- 172.21.15.64:0/2414707825 learned_addr learned my addr 172.21.15.64:0/2414707825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.019+0000 7f86cf7fe700 1 -- 172.21.15.64:0/2414707825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94caa80 msgr2=0x5631e94caea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:07:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.019+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94caa80 0x5631e94caea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.019+0000 7f86cf7fe700 1 -- 172.21.15.64:0/2414707825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 msgr2=0x5631e94cc750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:07:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.019+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cc750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.019+0000 7f86cf7fe700 1 -- 172.21.15.64:0/2414707825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e929c350 con 0x5631e94cb3e0 2024-05-08T22:07:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.020+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 0x5631e94cbcf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009840 tx=0x7f86b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f02e4a0b1d53d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.021+0000 7f86c5ffb700 1 -- 172.21.15.64:0/2414707825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8011830 con 0x5631e94cb3e0 2024-05-08T22:07:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.021+0000 7f86c5ffb700 1 -- 172.21.15.64:0/2414707825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b8011e50 con 0x5631e94cb3e0 2024-05-08T22:07:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.021+0000 7f86c5ffb700 1 -- 172.21.15.64:0/2414707825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8010380 con 0x5631e94cb3e0 2024-05-08T22:07:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.021+0000 7f86ccff9700 1 -- 172.21.15.64:0/38576283 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f86c000f460 con 0x7f86d0104330 2024-05-08T22:07:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.022+0000 7f86e23c2b80 1 -- 172.21.15.64:0/2414707825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 msgr2=0x5631e94cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.022+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 0x5631e94cbcf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009840 tx=0x7f86b8009b50 comp rx=0 tx=0).stop 2024-05-08T22:07:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.023+0000 7f86ccff9700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f86bc0a1220 0x7f86bc0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.023+0000 7f86ccff9700 1 -- 172.21.15.64:0/38576283 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86c00cd840 con 0x7f86d0104330 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.024+0000 7f86e23c2b80 1 -- 172.21.15.64:0/2414707825 shutdown_connections 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.024+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.024+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94caa80 0x5631e94caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.024+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/2414707825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cb3e0 0x5631e94cbcf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.024+0000 7f86e23c2b80 1 -- 172.21.15.64:0/2414707825 >> 172.21.15.64:0/2414707825 conn(0x5631e94291b0 msgr2=0x5631e94c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.026+0000 7f86cffff700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f86bc0a1220 0x7f86bc0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.026+0000 7f86cffff700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f86bc0a1220 0x7f86bc0a36e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e350 tx=0x7f86c800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.029+0000 7f86e23c2b80 1 -- 172.21.15.64:0/2414707825 shutdown_connections 2024-05-08T22:07:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.030+0000 7f86e23c2b80 1 -- 172.21.15.64:0/2414707825 wait complete. 2024-05-08T22:07:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.030+0000 7f86e23c2b80 1 -- start start 2024-05-08T22:07:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.030+0000 7f86e23c2b80 1 Processor -- start 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cb3e0 0x5631e94d4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cc2d0 0x5631e94dbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e93b7500 con 0x5631e94caa80 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e93b71a0 con 0x5631e94cb3e0 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e929e640 con 0x5631e94cc2d0 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86e23c2b80 10 client.0 _ll_register_callbacks cb 0x5631e93c0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60716/0 (socket says 172.21.15.64:60716) 2024-05-08T22:07:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.031+0000 7f86cf7fe700 1 -- 172.21.15.64:0/152794184 learned_addr learned my addr 172.21.15.64:0/152794184 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.032+0000 7f86cf7fe700 1 -- 172.21.15.64:0/152794184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cc2d0 msgr2=0x5631e94dbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.032+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cc2d0 0x5631e94dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.032+0000 7f86cf7fe700 1 -- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cb3e0 msgr2=0x5631e94d4ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.032+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cb3e0 0x5631e94d4ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.032+0000 7f86cf7fe700 1 -- 172.21.15.64:0/152794184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b8009470 con 0x5631e94caa80 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.033+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94d4980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009fd0 tx=0x7f86b800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a993c6fda3987c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.033+0000 7f86b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:07:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.033+0000 7f86b6ffd700 1 -- 172.21.15.64:0/152794184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8011830 con 0x5631e94caa80 2024-05-08T22:07:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.033+0000 7f86b6ffd700 1 -- 172.21.15.64:0/152794184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86b8011990 con 0x5631e94caa80 2024-05-08T22:07:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.033+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631e9374210 con 0x5631e94caa80 2024-05-08T22:07:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86b6ffd700 1 -- 172.21.15.64:0/152794184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86b8011b50 con 0x5631e94caa80 2024-05-08T22:07:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 10 client.16077 fetch_fsmap learned FSMap version 46 2024-05-08T22:07:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5631e9248b00 con 0x5631e94caa80 2024-05-08T22:07:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86b6ffd700 1 -- 172.21.15.64:0/152794184 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631e9374210 con 0x5631e94caa80 2024-05-08T22:07:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86b6ffd700 1 -- 172.21.15.64:0/152794184 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f86b8022030 con 0x5631e94caa80 2024-05-08T22:07:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:07:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 20 client.16077 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e94e0720 0x5631e94e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 4 client.16077 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.034+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631e949a7b0 con 0x5631e94e0720 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.035+0000 7f86ceffd700 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e94e0720 0x5631e94e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.035+0000 7f86b6ffd700 10 client.16077 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.035+0000 7f86ceffd700 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e94e0720 0x5631e94e2b40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436dcaa4ac70d1d server_cookie=628d3c68c2be9d0a in_seq=0 out_seq=0 2024-05-08T22:07:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.036+0000 7f86b6ffd700 1 -- 172.21.15.64:0/152794184 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631e949a7b0 con 0x5631e94e0720 2024-05-08T22:07:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.036+0000 7f86b6ffd700 10 client.16077 handle_command_reply: tid=0 2024-05-08T22:07:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.036+0000 7f86e23c2b80 1 client.16077 shutdown 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 msgr2=0x5631e94d4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94d4980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009fd0 tx=0x7f86b800bf70 comp rx=0 tx=0).stop 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 shutdown_connections 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e94e0720 0x5631e94e2b40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cc2d0 0x5631e94dbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94d4980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/152794184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cb3e0 0x5631e94d4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 >> 172.21.15.64:0/152794184 conn(0x5631e94291b0 msgr2=0x5631e94c5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86b6ffd700 0 client.16077 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 shutdown_connections 2024-05-08T22:07:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.042+0000 7f86e23c2b80 1 -- 172.21.15.64:0/152794184 wait complete. 2024-05-08T22:07:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.043+0000 7f86e23c2b80 20 client.16077 trim_cache size 0 max 16384 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 Processor -- start 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 -- start start 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cb3e0 0x5631e94dbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94c6930 con 0x5631e94caa80 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94d3390 con 0x5631e94cc2d0 2024-05-08T22:07:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94d5b70 con 0x5631e94cb3e0 2024-05-08T22:07:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60728/0 (socket says 172.21.15.64:60728) 2024-05-08T22:07:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86cf7fe700 1 -- 172.21.15.64:0/1783268909 learned_addr learned my addr 172.21.15.64:0/1783268909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.062+0000 7f86ceffd700 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cb3e0 0x5631e94dbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cffff700 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cf7fe700 1 -- 172.21.15.64:0/1783268909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cb3e0 msgr2=0x5631e94dbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cb3e0 0x5631e94dbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cf7fe700 1 -- 172.21.15.64:0/1783268909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 msgr2=0x5631e94cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cf7fe700 1 -- 172.21.15.64:0/1783268909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e93e1500 con 0x5631e94caa80 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94cc6b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f86b8000c00 tx=0x7f86b8009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aa1ad72ce863bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86c67fc700 1 -- 172.21.15.64:0/1783268909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8009140 con 0x5631e94caa80 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.063+0000 7f86c67fc700 1 -- 172.21.15.64:0/1783268909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b801e070 con 0x5631e94caa80 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.064+0000 7f86c67fc700 1 -- 172.21.15.64:0/1783268909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8010a00 con 0x5631e94caa80 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.064+0000 7f86e23c2b80 1 -- 172.21.15.64:0/1783268909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 msgr2=0x5631e94cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.064+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94cc6b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f86b8000c00 tx=0x7f86b8009740 comp rx=0 tx=0).stop 2024-05-08T22:07:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.065+0000 7f86e23c2b80 1 -- 172.21.15.64:0/1783268909 shutdown_connections 2024-05-08T22:07:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.065+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94cc2d0 0x5631e94cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.065+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94caa80 0x5631e94cc6b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.065+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/1783268909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94cb3e0 0x5631e94dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.065+0000 7f86e23c2b80 1 -- 172.21.15.64:0/1783268909 >> 172.21.15.64:0/1783268909 conn(0x5631e94291b0 msgr2=0x5631e94d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.077+0000 7f86e23c2b80 1 -- 172.21.15.64:0/1783268909 shutdown_connections 2024-05-08T22:07:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.078+0000 7f86e23c2b80 1 -- 172.21.15.64:0/1783268909 wait complete. 2024-05-08T22:07:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.078+0000 7f86e23c2b80 1 -- start start 2024-05-08T22:07:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.078+0000 7f86e23c2b80 1 Processor -- start 2024-05-08T22:07:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94caa80 0x5631e94d97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cc2d0 0x5631e94d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 0x5631e94db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94db600 con 0x5631e94cc2d0 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94db770 con 0x5631e94caa80 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e94db930 con 0x5631e94dac40 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86e23c2b80 10 client.0 _ll_register_callbacks cb 0x5631e93c0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 0x5631e94db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 0x5631e94db0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:56950/0 (socket says 172.21.15.64:56950) 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 -- 172.21.15.64:0/501806466 learned_addr learned my addr 172.21.15.64:0/501806466 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86ceffd700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cc2d0 0x5631e94d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94caa80 0x5631e94d97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 -- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94caa80 msgr2=0x5631e94d97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94caa80 0x5631e94d97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 -- 172.21.15.64:0/501806466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cc2d0 msgr2=0x5631e94d9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cc2d0 0x5631e94d9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cffff700 1 -- 172.21.15.64:0/501806466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631e9248b00 con 0x5631e94dac40 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.079+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94caa80 0x5631e94d97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.080+0000 7f86cffff700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 0x5631e94db0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86c8015df0 tx=0x7f86c8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6263f65c97f83968 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.080+0000 7f86b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:07:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.080+0000 7f86b6ffd700 1 -- 172.21.15.64:0/501806466 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800ee60 con 0x5631e94dac40 2024-05-08T22:07:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.080+0000 7f86b6ffd700 1 -- 172.21.15.64:0/501806466 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86c8013ce0 con 0x5631e94dac40 2024-05-08T22:07:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.080+0000 7f86ceffd700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cc2d0 0x5631e94d9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.080+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86b8011e50 con 0x5631e94dac40 2024-05-08T22:07:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.081+0000 7f86b6ffd700 1 -- 172.21.15.64:0/501806466 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c8013ea0 con 0x5631e94dac40 2024-05-08T22:07:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.081+0000 7f86e23c2b80 10 client.25708 fetch_fsmap learned FSMap version 46 2024-05-08T22:07:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.081+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5631e92a0c70 con 0x5631e94dac40 2024-05-08T22:07:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86b6ffd700 1 -- 172.21.15.64:0/501806466 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b8011e50 con 0x5631e94dac40 2024-05-08T22:07:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86b6ffd700 1 -- 172.21.15.64:0/501806466 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f86c802a6d0 con 0x5631e94dac40 2024-05-08T22:07:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86e23c2b80 10 client.25708 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:07:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86e23c2b80 10 client.25708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86e23c2b80 20 client.25708 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e9537c10 0x5631e953a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86e23c2b80 4 client.25708 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:07:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.082+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631e9413e60 con 0x5631e9537c10 2024-05-08T22:07:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.098+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e9537c10 0x5631e953a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.116+0000 7f86cf7fe700 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e9537c10 0x5631e953a030 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cb81a9282f4688 server_cookie=9447304b4a4d7f5 in_seq=0 out_seq=0 2024-05-08T22:07:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.116+0000 7f86b6ffd700 10 client.25708 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.153+0000 7f86b6ffd700 1 -- 172.21.15.64:0/501806466 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631e9413e60 con 0x5631e9537c10 2024-05-08T22:07:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.153+0000 7f86b6ffd700 10 client.25708 handle_command_reply: tid=0 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 client.25708 shutdown 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 msgr2=0x5631e94db0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 0x5631e94db0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86c8015df0 tx=0x7f86c8009920 comp rx=0 tx=0).stop 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 shutdown_connections 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5631e94caa80 0x5631e94d97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5631e9537c10 0x5631e953a030 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631e94dac40 0x5631e94db0c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 --2- 172.21.15.64:0/501806466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631e94cc2d0 0x5631e94d9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86b6ffd700 0 client.25708 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 >> 172.21.15.64:0/501806466 conn(0x5631e94291b0 msgr2=0x5631e94cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 shutdown_connections 2024-05-08T22:07:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 1 -- 172.21.15.64:0/501806466 wait complete. 2024-05-08T22:07:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.156+0000 7f86e23c2b80 20 client.25708 trim_cache size 0 max 16384 2024-05-08T22:07:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:07:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:07:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:07:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f86bc0a1220 msgr2=0x7f86bc0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f86bc0a1220 0x7f86bc0a36e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e350 tx=0x7f86c800b040 comp rx=0 tx=0).stop 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 msgr2=0x7f86d01952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d01952b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005f50 tx=0x7f86c0003ae0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 shutdown_connections 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f86bc0a1220 0x7f86bc0a36e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86d0104cd0 0x7f86d019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d0103140 0x7f86d0194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 --2- 172.21.15.64:0/38576283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0104330 0x7f86d01952b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 >> 172.21.15.64:0/38576283 conn(0x7f86d00fe6b0 msgr2=0x7f86d00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 shutdown_connections 2024-05-08T22:07:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:01.157+0000 7f86d6208700 1 -- 172.21.15.64:0/38576283 wait complete. 2024-05-08T22:07:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 730.0 s, 1300.5 tps, lat 76.938 ms stddev 82.787 2024-05-08T22:07:01.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:01 smithi186 ceph-mon[46619]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 810 op/s 2024-05-08T22:07:01.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:01 smithi064 ceph-mon[42500]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 810 op/s 2024-05-08T22:07:02.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:01 smithi160 ceph-mon[46962]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 810 op/s 2024-05-08T22:07:03.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:03 smithi186 ceph-mon[46619]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 660 op/s 2024-05-08T22:07:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:03 smithi064 ceph-mon[42500]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 660 op/s 2024-05-08T22:07:04.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:03 smithi160 ceph-mon[46962]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 660 op/s 2024-05-08T22:07:05.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:05 smithi064 ceph-mon[42500]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 635 op/s 2024-05-08T22:07:06.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:05 smithi160 ceph-mon[46962]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 635 op/s 2024-05-08T22:07:06.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:05 smithi186 ceph-mon[46619]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 635 op/s 2024-05-08T22:07:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 735.0 s, 1238.4 tps, lat 81.406 ms stddev 90.121 2024-05-08T22:07:06.973 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:06 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:07:07.051 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:06 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:07:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:06 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:07:07.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:07 smithi064 ceph-mon[42500]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 730 op/s 2024-05-08T22:07:08.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:07 smithi160 ceph-mon[46962]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 730 op/s 2024-05-08T22:07:08.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:07 smithi186 ceph-mon[46619]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 730 op/s 2024-05-08T22:07:08.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:08 smithi064 ceph-mon[42500]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 684 op/s 2024-05-08T22:07:09.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:08 smithi160 ceph-mon[46962]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 684 op/s 2024-05-08T22:07:09.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:08 smithi186 ceph-mon[46619]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 684 op/s 2024-05-08T22:07:11.479 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 740.0 s, 900.8 tps, lat 109.850 ms stddev 160.564 2024-05-08T22:07:11.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:11 smithi186 ceph-mon[46619]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 MiB/s wr, 1.55k op/s 2024-05-08T22:07:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:11 smithi064 ceph-mon[42500]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 MiB/s wr, 1.55k op/s 2024-05-08T22:07:12.050 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:11 smithi160 ceph-mon[46962]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 MiB/s wr, 1.55k op/s 2024-05-08T22:07:13.895 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:13 smithi186 ceph-mon[46619]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.88k op/s 2024-05-08T22:07:13.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:13 smithi064 ceph-mon[42500]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.88k op/s 2024-05-08T22:07:14.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:13 smithi160 ceph-mon[46962]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.88k op/s 2024-05-08T22:07:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.347+0000 7efe00609700 1 -- 172.21.15.64:0/3755686468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8108730 msgr2=0x7efdf8108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.347+0000 7efe00609700 1 --2- 172.21.15.64:0/3755686468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8108730 0x7efdf8108b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdf40099c0 tx=0x7efdf4009cd0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 -- 172.21.15.64:0/3755686468 shutdown_connections 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 --2- 172.21.15.64:0/3755686468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 0x7efdf810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 --2- 172.21.15.64:0/3755686468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf8107540 0x7efdf8107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 --2- 172.21.15.64:0/3755686468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8108730 0x7efdf8108b90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 -- 172.21.15.64:0/3755686468 >> 172.21.15.64:0/3755686468 conn(0x7efdf80764d0 msgr2=0x7efdf8078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 -- 172.21.15.64:0/3755686468 shutdown_connections 2024-05-08T22:07:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 -- 172.21.15.64:0/3755686468 wait complete. 2024-05-08T22:07:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.348+0000 7efe00609700 1 Processor -- start 2024-05-08T22:07:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 -- start start 2024-05-08T22:07:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 0x7efdf819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf8108730 0x7efdf819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 0x7efdf81a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf81a5000 con 0x7efdf81090d0 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf806a9c0 con 0x7efdf8107540 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efe00609700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf806ab50 con 0x7efdf8108730 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 0x7efdf819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 0x7efdf819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35486/0 (socket says 172.21.15.64:35486) 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 -- 172.21.15.64:0/3590214216 learned_addr learned my addr 172.21.15.64:0/3590214216 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 -- 172.21.15.64:0/3590214216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf8108730 msgr2=0x7efdf819da10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf8108730 0x7efdf819da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 -- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 msgr2=0x7efdf81a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfeba6700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 0x7efdf81a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 0x7efdf81a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.349+0000 7efdfe3a5700 1 -- 172.21.15.64:0/3590214216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40095f0 con 0x7efdf8107540 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efdfeba6700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 0x7efdf81a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 0x7efdf819d4d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efde800e970 tx=0x7efde800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efdef7fe700 1 -- 172.21.15.64:0/3590214216 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8009790 con 0x7efdf8107540 2024-05-08T22:07:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efdef7fe700 1 -- 172.21.15.64:0/3590214216 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde80044e0 con 0x7efdf8107540 2024-05-08T22:07:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efdef7fe700 1 -- 172.21.15.64:0/3590214216 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8003e30 con 0x7efdf8107540 2024-05-08T22:07:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf806ae30 con 0x7efdf8107540 2024-05-08T22:07:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.350+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf806b2f0 con 0x7efdf8107540 2024-05-08T22:07:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.353+0000 7efdef7fe700 1 -- 172.21.15.64:0/3590214216 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efde8003f90 con 0x7efdf8107540 2024-05-08T22:07:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.354+0000 7efdef7fe700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efde40a1270 0x7efde40a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.354+0000 7efdef7fe700 1 -- 172.21.15.64:0/3590214216 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efde8014070 con 0x7efdf8107540 2024-05-08T22:07:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.354+0000 7efdfdba4700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efde40a1270 0x7efde40a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efdfdba4700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efde40a1270 0x7efde40a3730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009fd0 tx=0x7efdf400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 Processor -- start 2024-05-08T22:07:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 -- start start 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e15f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e15f3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e15490f0 con 0x55b2e15f32d0 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1546ab0 con 0x55b2e15f23e0 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.355+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e15423a0 con 0x55b2e15f1a80 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:38430/0 (socket says 172.21.15.64:38430) 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 -- 172.21.15.64:0/1117789643 learned_addr learned my addr 172.21.15.64:0/1117789643 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfeba6700 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e15f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfdba4700 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e15f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 -- 172.21.15.64:0/1117789643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 msgr2=0x55b2e15f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e15f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 -- 172.21.15.64:0/1117789643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 msgr2=0x55b2e15f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e15f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 -- 172.21.15.64:0/1117789643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2e13c3350 con 0x55b2e15f1a80 2024-05-08T22:07:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.356+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f1ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efde80a1e50 tx=0x7efde8099a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141c39ecc1cd5bca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.360+0000 7efdecff9700 1 -- 172.21.15.64:0/1117789643 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde809a600 con 0x55b2e15f1a80 2024-05-08T22:07:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.360+0000 7efdecff9700 1 -- 172.21.15.64:0/1117789643 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde8099f00 con 0x55b2e15f1a80 2024-05-08T22:07:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.360+0000 7efdecff9700 1 -- 172.21.15.64:0/1117789643 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8004240 con 0x55b2e15f1a80 2024-05-08T22:07:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1117789643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 msgr2=0x55b2e15f1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f1ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efde80a1e50 tx=0x7efde8099a40 comp rx=0 tx=0).stop 2024-05-08T22:07:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1117789643 shutdown_connections 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e15f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f1ea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1117789643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e15f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1117789643 >> 172.21.15.64:0/1117789643 conn(0x55b2e15501b0 msgr2=0x55b2e15ee440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1117789643 shutdown_connections 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.361+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1117789643 wait complete. 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 -- start start 2024-05-08T22:07:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 Processor -- start 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f1a80 0x55b2e15fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 0x55b2e15fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f32d0 0x55b2e1602ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e14de500 con 0x55b2e15f23e0 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e14de1a0 con 0x55b2e15f1a80 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e13c5640 con 0x55b2e15f32d0 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.362+0000 7efe0c7c3b80 10 client.0 _ll_register_callbacks cb 0x55b2e14e7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfdba4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 0x55b2e15fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfdba4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 0x55b2e15fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49112/0 (socket says 172.21.15.64:49112) 2024-05-08T22:07:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfdba4700 1 -- 172.21.15.64:0/1694864858 learned_addr learned my addr 172.21.15.64:0/1694864858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f32d0 0x55b2e1602ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f1a80 0x55b2e15fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 -- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f1a80 msgr2=0x55b2e15fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f1a80 0x55b2e15fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 -- 172.21.15.64:0/1694864858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 msgr2=0x55b2e15fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 0x55b2e15fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 -- 172.21.15.64:0/1694864858 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf806b2f0 con 0x55b2e15f32d0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfdba4700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 0x55b2e15fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efdfeba6700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f32d0 0x55b2e1602ee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55b2e15f2ab0 tx=0x7efdf0007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec68ae846794046f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.363+0000 7efde1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efde1ffb700 1 -- 172.21.15.64:0/1694864858 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003c10 con 0x55b2e15f32d0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efde1ffb700 1 -- 172.21.15.64:0/1694864858 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efdf0003d70 con 0x55b2e15f32d0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2e149b210 con 0x55b2e15f32d0 2024-05-08T22:07:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efde1ffb700 1 -- 172.21.15.64:0/1694864858 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf0011690 con 0x55b2e15f32d0 2024-05-08T22:07:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 10 client.25717 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55b2e136fb00 con 0x55b2e15f32d0 2024-05-08T22:07:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efde1ffb700 1 -- 172.21.15.64:0/1694864858 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2e149b210 con 0x55b2e15f32d0 2024-05-08T22:07:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efde1ffb700 1 -- 172.21.15.64:0/1694864858 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efdf0014070 con 0x55b2e15f32d0 2024-05-08T22:07:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 10 client.25717 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 20 client.25717 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e1607820 0x55b2e1609c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 4 client.25717 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.364+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2e15c17b0 con 0x55b2e1607820 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.365+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e1607820 0x55b2e1609c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.365+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e1607820 0x55b2e1609c40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9959c0e541a99baf server_cookie=f7e07dd9874e7dc4 in_seq=0 out_seq=0 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.365+0000 7efde1ffb700 10 client.25717 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efde1ffb700 1 -- 172.21.15.64:0/1694864858 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2e15c17b0 con 0x55b2e1607820 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efde1ffb700 10 client.25717 handle_command_reply: tid=0 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 client.25717 shutdown 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f32d0 msgr2=0x55b2e1602ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f32d0 0x55b2e1602ee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55b2e15f2ab0 tx=0x7efdf0007dd0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 shutdown_connections 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f1a80 0x55b2e15fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e1607820 0x55b2e1609c40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f32d0 0x55b2e1602ee0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1694864858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f23e0 0x55b2e15fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 >> 172.21.15.64:0/1694864858 conn(0x55b2e15501b0 msgr2=0x55b2e15ec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efde1ffb700 0 client.25717 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 shutdown_connections 2024-05-08T22:07:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1694864858 wait complete. 2024-05-08T22:07:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.366+0000 7efe0c7c3b80 20 client.25717 trim_cache size 0 max 16384 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 Processor -- start 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 -- start start 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e1602ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e1603590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e15f2bb0 con 0x55b2e15f32d0 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e15e77f0 con 0x55b2e15f23e0 2024-05-08T22:07:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.390+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1603ad0 con 0x55b2e15f1a80 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfeba6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e1603590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e1602ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfeba6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e1603590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49128/0 (socket says 172.21.15.64:49128) 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfeba6700 1 -- 172.21.15.64:0/1133129911 learned_addr learned my addr 172.21.15.64:0/1133129911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 -- 172.21.15.64:0/1133129911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 msgr2=0x55b2e15f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 -- 172.21.15.64:0/1133129911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 msgr2=0x55b2e1603590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e1603590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 -- 172.21.15.64:0/1133129911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2e1508500 con 0x55b2e15f23e0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efdfdba4700 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e1602ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006010 tx=0x7efdf401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=887ccf72523da47f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efded7fa700 1 -- 172.21.15.64:0/1133129911 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf401c070 con 0x55b2e15f23e0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efded7fa700 1 -- 172.21.15.64:0/1133129911 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf4029040 con 0x55b2e15f23e0 2024-05-08T22:07:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.391+0000 7efded7fa700 1 -- 172.21.15.64:0/1133129911 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4003b50 con 0x55b2e15f23e0 2024-05-08T22:07:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1133129911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 msgr2=0x55b2e1602ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e1602ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006010 tx=0x7efdf401c000 comp rx=0 tx=0).stop 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1133129911 shutdown_connections 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e1603590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e15f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/1133129911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e1602ea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1133129911 >> 172.21.15.64:0/1133129911 conn(0x55b2e15501b0 msgr2=0x55b2e15f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1133129911 shutdown_connections 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.392+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/1133129911 wait complete. 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 -- start start 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 Processor -- start 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e1600180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e16006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e16016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1601be0 con 0x55b2e15f32d0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1601d50 con 0x55b2e15f23e0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1601f10 con 0x55b2e15f1a80 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.393+0000 7efe0c7c3b80 10 client.0 _ll_register_callbacks cb 0x55b2e14e7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e16006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e16006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:35520/0 (socket says 172.21.15.64:35520) 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 -- 172.21.15.64:0/3118420207 learned_addr learned my addr 172.21.15.64:0/3118420207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e1600180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfeba6700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e16016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 -- 172.21.15.64:0/3118420207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 msgr2=0x55b2e1600180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e1600180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 -- 172.21.15.64:0/3118420207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 msgr2=0x55b2e16016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e16016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 -- 172.21.15.64:0/3118420207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf81102d0 con 0x55b2e15f23e0 2024-05-08T22:07:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfeba6700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e16016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:07:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efdfdba4700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e16006c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efdf401e040 tx=0x7efdf400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862129b33deefbde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efde1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:07:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.394+0000 7efde1ffb700 1 -- 172.21.15.64:0/3118420207 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40053c0 con 0x55b2e15f23e0 2024-05-08T22:07:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.395+0000 7efde1ffb700 1 -- 172.21.15.64:0/3118420207 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efdf40038d0 con 0x55b2e15f23e0 2024-05-08T22:07:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.395+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efde8099f00 con 0x55b2e15f23e0 2024-05-08T22:07:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.395+0000 7efde1ffb700 1 -- 172.21.15.64:0/3118420207 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf4005520 con 0x55b2e15f23e0 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 10 client.25880 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55b2e13c7c70 con 0x55b2e15f23e0 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efde1ffb700 1 -- 172.21.15.64:0/3118420207 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde8099f00 con 0x55b2e15f23e0 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efde1ffb700 1 -- 172.21.15.64:0/3118420207 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efdf4040070 con 0x55b2e15f23e0 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 10 client.25880 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 20 client.25880 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e165dd60 0x55b2e1660150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 4 client.25880 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:07:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b2e153ae60 con 0x55b2e165dd60 2024-05-08T22:07:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.396+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e165dd60 0x55b2e1660150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.397+0000 7efde1ffb700 10 client.25880 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.397+0000 7efdfe3a5700 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e165dd60 0x55b2e1660150 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=683eea98d43d8cc1 server_cookie=f1b8de64f941c247 in_seq=0 out_seq=0 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.407+0000 7efde1ffb700 1 -- 172.21.15.64:0/3118420207 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b2e153ae60 con 0x55b2e165dd60 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.407+0000 7efde1ffb700 10 client.25880 handle_command_reply: tid=0 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.407+0000 7efe0c7c3b80 1 client.25880 shutdown 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 msgr2=0x55b2e16006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e16006c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efdf401e040 tx=0x7efdf400ff90 comp rx=0 tx=0).stop 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 shutdown_connections 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b2e165dd60 0x55b2e1660150 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2e15f32d0 0x55b2e16016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2e15f1a80 0x55b2e1600180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 --2- 172.21.15.64:0/3118420207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2e15f23e0 0x55b2e16006c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 >> 172.21.15.64:0/3118420207 conn(0x55b2e15501b0 msgr2=0x55b2e160e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efde1ffb700 0 client.25880 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 shutdown_connections 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 1 -- 172.21.15.64:0/3118420207 wait complete. 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe0c7c3b80 20 client.25880 trim_cache size 0 max 16384 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:07:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:07:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efde40a1270 msgr2=0x7efde40a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efde40a1270 0x7efde40a3730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009fd0 tx=0x7efdf400b540 comp rx=0 tx=0).stop 2024-05-08T22:07:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 msgr2=0x7efdf819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 0x7efdf819d4d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efde800e970 tx=0x7efde800ed30 comp rx=0 tx=0).stop 2024-05-08T22:07:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 shutdown_connections 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7efde40a1270 0x7efde40a3730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdf8107540 0x7efdf819d4d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdf81090d0 0x7efdf81a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 --2- 172.21.15.64:0/3590214216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf8108730 0x7efdf819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.408+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 >> 172.21.15.64:0/3590214216 conn(0x7efdf80764d0 msgr2=0x7efdf810bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.409+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 shutdown_connections 2024-05-08T22:07:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:14.409+0000 7efe00609700 1 -- 172.21.15.64:0/3590214216 wait complete. 2024-05-08T22:07:15.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:15 smithi186 ceph-mon[46619]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 1.82k op/s 2024-05-08T22:07:15.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:15 smithi064 ceph-mon[42500]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 1.82k op/s 2024-05-08T22:07:16.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:15 smithi160 ceph-mon[46962]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 1.82k op/s 2024-05-08T22:07:16.482 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 745.0 s, 1514.6 tps, lat 66.708 ms stddev 73.195 2024-05-08T22:07:17.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:17 smithi186 ceph-mon[46619]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.88k op/s 2024-05-08T22:07:17.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:17 smithi064 ceph-mon[42500]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.88k op/s 2024-05-08T22:07:18.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:17 smithi160 ceph-mon[46962]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.88k op/s 2024-05-08T22:07:19.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:19 smithi064 ceph-mon[42500]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-05-08T22:07:20.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:19 smithi160 ceph-mon[46962]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-05-08T22:07:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:19 smithi186 ceph-mon[46619]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-05-08T22:07:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 750.0 s, 1575.5 tps, lat 63.780 ms stddev 71.578 2024-05-08T22:07:21.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:21 smithi064 ceph-mon[42500]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.85k op/s 2024-05-08T22:07:22.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:21 smithi160 ceph-mon[46962]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.85k op/s 2024-05-08T22:07:22.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:21 smithi186 ceph-mon[46619]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.85k op/s 2024-05-08T22:07:23.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:23 smithi064 ceph-mon[42500]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-05-08T22:07:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:23 smithi160 ceph-mon[46962]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-05-08T22:07:24.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:23 smithi186 ceph-mon[46619]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-05-08T22:07:25.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:25 smithi064 ceph-mon[42500]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 582 op/s 2024-05-08T22:07:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:25 smithi160 ceph-mon[46962]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 582 op/s 2024-05-08T22:07:26.146 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:25 smithi186 ceph-mon[46619]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 582 op/s 2024-05-08T22:07:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 755.0 s, 1487.4 tps, lat 66.697 ms stddev 71.770 2024-05-08T22:07:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 -- 172.21.15.64:0/1964804445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1090d0 msgr2=0x7f6b2c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 --2- 172.21.15.64:0/1964804445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1090d0 0x7f6b2c10f980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009920 tx=0x7f6b1c009c30 comp rx=0 tx=0).stop 2024-05-08T22:07:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 -- 172.21.15.64:0/1964804445 shutdown_connections 2024-05-08T22:07:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 --2- 172.21.15.64:0/1964804445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1090d0 0x7f6b2c10f980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 --2- 172.21.15.64:0/1964804445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 --2- 172.21.15.64:0/1964804445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c108730 0x7f6b2c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.590+0000 7f6b33772700 1 -- 172.21.15.64:0/1964804445 >> 172.21.15.64:0/1964804445 conn(0x7f6b2c0764d0 msgr2=0x7f6b2c078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.601+0000 7f6b33772700 1 -- 172.21.15.64:0/1964804445 shutdown_connections 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.603+0000 7f6b33772700 1 -- 172.21.15.64:0/1964804445 wait complete. 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.603+0000 7f6b33772700 1 Processor -- start 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.603+0000 7f6b33772700 1 -- start start 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b33772700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c07e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b33772700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c108730 0x7f6b2c07eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b33772700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c1090d0 0x7f6b2c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b33772700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c081ca0 con 0x7f6b2c108730 2024-05-08T22:07:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b33772700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c081e10 con 0x7f6b2c1090d0 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b33772700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c19cc00 con 0x7f6b2c107540 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c07e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c07e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:53558/0 (socket says 172.21.15.64:53558) 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 -- 172.21.15.64:0/2439506917 learned_addr learned my addr 172.21.15.64:0/2439506917 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c1090d0 0x7f6b2c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 -- 172.21.15.64:0/2439506917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c1090d0 msgr2=0x7f6b2c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c1090d0 0x7f6b2c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 -- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c108730 msgr2=0x7f6b2c07eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c108730 0x7f6b2c07eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 -- 172.21.15.64:0/2439506917 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009520 con 0x7f6b2c107540 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.604+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c07e660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400efb0 tx=0x7f6b2400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.605+0000 7f6b227fc700 1 -- 172.21.15.64:0/2439506917 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24018070 con 0x7f6b2c107540 2024-05-08T22:07:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.605+0000 7f6b227fc700 1 -- 172.21.15.64:0/2439506917 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b2400f040 con 0x7f6b2c107540 2024-05-08T22:07:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.605+0000 7f6b227fc700 1 -- 172.21.15.64:0/2439506917 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24004310 con 0x7f6b2c107540 2024-05-08T22:07:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.606+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c19ce80 con 0x7f6b2c107540 2024-05-08T22:07:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.606+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c19d310 con 0x7f6b2c107540 2024-05-08T22:07:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.609+0000 7f6b227fc700 1 -- 172.21.15.64:0/2439506917 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6b2400cad0 con 0x7f6b2c107540 2024-05-08T22:07:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.610+0000 7f6b227fc700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6b180a1270 0x7f6b180a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.611+0000 7f6b227fc700 1 -- 172.21.15.64:0/2439506917 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6b240cc2c0 con 0x7f6b2c107540 2024-05-08T22:07:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.611+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6b180a1270 0x7f6b180a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.611+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6b180a1270 0x7f6b180a3730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c07f730 tx=0x7f6b28006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 Processor -- start 2024-05-08T22:07:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 -- start start 2024-05-08T22:07:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a18ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a19cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a599700f0 con 0x561a59a18a80 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5996dab0 con 0x561a59a1a2d0 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.612+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a599693a0 con 0x561a59a193e0 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a18ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a18ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40374/0 (socket says 172.21.15.64:40374) 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 -- 172.21.15.64:0/63415782 learned_addr learned my addr 172.21.15.64:0/63415782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a19cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 -- 172.21.15.64:0/63415782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 msgr2=0x561a59a19cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a19cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 -- 172.21.15.64:0/63415782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 msgr2=0x561a59a1a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 -- 172.21.15.64:0/63415782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a597ea350 con 0x561a59a18a80 2024-05-08T22:07:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a19cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.613+0000 7f6b3150e700 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a18ea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b24096130 tx=0x7f6b24099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84580c254facc4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.616+0000 7f6b177fe700 1 -- 172.21.15.64:0/63415782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24099dc0 con 0x561a59a18a80 2024-05-08T22:07:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.616+0000 7f6b177fe700 1 -- 172.21.15.64:0/63415782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b240996c0 con 0x561a59a18a80 2024-05-08T22:07:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.616+0000 7f6b177fe700 1 -- 172.21.15.64:0/63415782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400cd80 con 0x561a59a18a80 2024-05-08T22:07:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.617+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/63415782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 msgr2=0x561a59a18ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.617+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a18ea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b24096130 tx=0x7f6b24099280 comp rx=0 tx=0).stop 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/63415782 shutdown_connections 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a18ea0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/63415782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a19cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/63415782 >> 172.21.15.64:0/63415782 conn(0x561a599771b0 msgr2=0x561a59a15440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/63415782 shutdown_connections 2024-05-08T22:07:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/63415782 wait complete. 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.618+0000 7f6b3f92cb80 1 -- start start 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 Processor -- start 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a22e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59905500 con 0x561a59a1a2d0 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a599051a0 con 0x561a59a18a80 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a597ec640 con 0x561a59a193e0 2024-05-08T22:07:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.619+0000 7f6b3f92cb80 10 client.0 _ll_register_callbacks cb 0x561a5990ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b31d0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b31d0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40376/0 (socket says 172.21.15.64:40376) 2024-05-08T22:07:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b31d0f700 1 -- 172.21.15.64:0/1743510796 learned_addr learned my addr 172.21.15.64:0/1743510796 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a22e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b31d0f700 1 -- 172.21.15.64:0/1743510796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 msgr2=0x561a59a22e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.621+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a22e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.622+0000 7f6b31d0f700 1 -- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 msgr2=0x561a59a228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.622+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.622+0000 7f6b31d0f700 1 -- 172.21.15.64:0/1743510796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c1102d0 con 0x561a59a1a2d0 2024-05-08T22:07:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.622+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009230 tx=0x7f6b1c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eabf61a9ca5b864 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:07:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b14ff9700 1 -- 172.21.15.64:0/1743510796 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c01d050 con 0x561a59a1a2d0 2024-05-08T22:07:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b14ff9700 1 -- 172.21.15.64:0/1743510796 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6b1c00bbc0 con 0x561a59a1a2d0 2024-05-08T22:07:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a598c2210 con 0x561a59a1a2d0 2024-05-08T22:07:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b14ff9700 1 -- 172.21.15.64:0/1743510796 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b1c005240 con 0x561a59a1a2d0 2024-05-08T22:07:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b3f92cb80 10 client.16104 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f6b2c19d310 con 0x561a59a1a2d0 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.624+0000 7f6b14ff9700 1 -- 172.21.15.64:0/1743510796 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a598c2210 con 0x561a59a1a2d0 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b14ff9700 1 -- 172.21.15.64:0/1743510796 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6b1c015020 con 0x561a59a1a2d0 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3f92cb80 10 client.16104 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3f92cb80 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3f92cb80 20 client.16104 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a3f7a0 0x561a59a41bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3f92cb80 4 client.16104 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a599e87b0 con 0x561a59a3f7a0 2024-05-08T22:07:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.625+0000 7f6b3150e700 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a3f7a0 0x561a59a41bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.627+0000 7f6b3150e700 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a3f7a0 0x561a59a41bc0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331155a121d4db38 server_cookie=bfeb19ca482ae616 in_seq=0 out_seq=0 2024-05-08T22:07:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.627+0000 7f6b14ff9700 10 client.16104 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.627+0000 7f6b14ff9700 1 -- 172.21.15.64:0/1743510796 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a599e87b0 con 0x561a59a3f7a0 2024-05-08T22:07:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.627+0000 7f6b14ff9700 10 client.16104 handle_command_reply: tid=0 2024-05-08T22:07:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 client.16104 shutdown 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 msgr2=0x561a59a29e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009230 tx=0x7f6b1c00b790 comp rx=0 tx=0).stop 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 shutdown_connections 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a3f7a0 0x561a59a41bc0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29e40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/1743510796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a22e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 >> 172.21.15.64:0/1743510796 conn(0x561a599771b0 msgr2=0x561a59a13860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b14ff9700 0 client.16104 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 shutdown_connections 2024-05-08T22:07:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/1743510796 wait complete. 2024-05-08T22:07:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.628+0000 7f6b3f92cb80 20 client.16104 trim_cache size 0 max 16384 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 Processor -- start 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 -- start start 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a21dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59a15a70 con 0x561a59a18a80 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59a1a150 con 0x561a59a1a2d0 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59a237c0 con 0x561a59a193e0 2024-05-08T22:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a21dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a21dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40382/0 (socket says 172.21.15.64:40382) 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.649+0000 7f6b3150e700 1 -- 172.21.15.64:0/2065129691 learned_addr learned my addr 172.21.15.64:0/2065129691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b3150e700 1 -- 172.21.15.64:0/2065129691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 msgr2=0x561a59a29e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b3150e700 1 -- 172.21.15.64:0/2065129691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 msgr2=0x561a59a1a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.650+0000 7f6b3150e700 1 -- 172.21.15.64:0/2065129691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a59796b00 con 0x561a59a18a80 2024-05-08T22:07:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.654+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a21dd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b24099650 tx=0x7f6b2401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcaaf3d8846e861 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.655+0000 7f6b17fff700 1 -- 172.21.15.64:0/2065129691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2409d4f0 con 0x561a59a18a80 2024-05-08T22:07:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.655+0000 7f6b17fff700 1 -- 172.21.15.64:0/2065129691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b24003a70 con 0x561a59a18a80 2024-05-08T22:07:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.655+0000 7f6b17fff700 1 -- 172.21.15.64:0/2065129691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240b91a0 con 0x561a59a18a80 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2065129691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 msgr2=0x561a59a21dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a21dd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b24099650 tx=0x7f6b2401f040 comp rx=0 tx=0).stop 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2065129691 shutdown_connections 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a1a2d0 0x561a59a1a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a18a80 0x561a59a21dd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2065129691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2065129691 >> 172.21.15.64:0/2065129691 conn(0x561a599771b0 msgr2=0x561a59a207d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2065129691 shutdown_connections 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2065129691 wait complete. 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 -- start start 2024-05-08T22:07:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.656+0000 7f6b3f92cb80 1 Processor -- start 2024-05-08T22:07:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59a93dc0 con 0x561a59a1a2d0 2024-05-08T22:07:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59a93f00 con 0x561a59a18a80 2024-05-08T22:07:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a59a940c0 con 0x561a59a193e0 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3f92cb80 10 client.0 _ll_register_callbacks cb 0x561a5990ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3150e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a292e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:46230/0 (socket says 172.21.15.64:46230) 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b3150e700 1 -- 172.21.15.64:0/2739691886 learned_addr learned my addr 172.21.15.64:0/2739691886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.657+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3150e700 1 -- 172.21.15.64:0/2739691886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 msgr2=0x561a59a29820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3150e700 1 -- 172.21.15.64:0/2739691886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 msgr2=0x561a59a29d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3150e700 1 -- 172.21.15.64:0/2739691886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a597ea350 con 0x561a59a18a80 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b31d0f700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3150e700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a292e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b240a3040 tx=0x7f6b24004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36fb8090977dd6f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:07:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b14ff9700 1 -- 172.21.15.64:0/2739691886 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240a1b40 con 0x561a59a18a80 2024-05-08T22:07:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b14ff9700 1 -- 172.21.15.64:0/2739691886 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6b2409d4f0 con 0x561a59a18a80 2024-05-08T22:07:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.658+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b240996c0 con 0x561a59a18a80 2024-05-08T22:07:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.662+0000 7f6b14ff9700 1 -- 172.21.15.64:0/2739691886 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b24097a60 con 0x561a59a18a80 2024-05-08T22:07:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 10 client.25895 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561a5992f500 con 0x561a59a18a80 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b14ff9700 1 -- 172.21.15.64:0/2739691886 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b240996c0 con 0x561a59a18a80 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b14ff9700 1 -- 172.21.15.64:0/2739691886 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6b240a16b0 con 0x561a59a18a80 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 10 client.25895 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 10 client.25895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 20 client.25895 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a95640 0x561a59a97a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 4 client.25895 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.664+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a59961e60 con 0x561a59a95640 2024-05-08T22:07:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.665+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a95640 0x561a59a97a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.665+0000 7f6b30d0d700 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a95640 0x561a59a97a30 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c8e8c4f3327907 server_cookie=1740d6ade69971cd in_seq=0 out_seq=0 2024-05-08T22:07:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.665+0000 7f6b14ff9700 10 client.25895 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b14ff9700 1 -- 172.21.15.64:0/2739691886 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a59961e60 con 0x561a59a95640 2024-05-08T22:07:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b14ff9700 10 client.25895 handle_command_reply: tid=0 2024-05-08T22:07:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 client.25895 shutdown 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 msgr2=0x561a59a292e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a292e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b240a3040 tx=0x7f6b24004b60 comp rx=0 tx=0).stop 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 shutdown_connections 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a59a18a80 0x561a59a292e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561a59a95640 0x561a59a97a30 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a59a1a2d0 0x561a59a29d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 --2- 172.21.15.64:0/2739691886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a59a193e0 0x561a59a29820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b14ff9700 0 client.25895 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 >> 172.21.15.64:0/2739691886 conn(0x561a599771b0 msgr2=0x561a59a464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 shutdown_connections 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.674+0000 7f6b3f92cb80 1 -- 172.21.15.64:0/2739691886 wait complete. 2024-05-08T22:07:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b3f92cb80 20 client.25895 trim_cache size 0 max 16384 2024-05-08T22:07:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:07:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:07:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:07:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6b180a1270 msgr2=0x7f6b180a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6b180a1270 0x7f6b180a3730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c07f730 tx=0x7f6b28006cd0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 msgr2=0x7f6b2c07e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c07e660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400efb0 tx=0x7f6b2400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 shutdown_connections 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f6b180a1270 0x7f6b180a3730 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c1090d0 0x7f6b2c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b2c107540 0x7f6b2c07e660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 --2- 172.21.15.64:0/2439506917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c108730 0x7f6b2c07eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.675+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 >> 172.21.15.64:0/2439506917 conn(0x7f6b2c0764d0 msgr2=0x7f6b2c10bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.676+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 shutdown_connections 2024-05-08T22:07:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:27.676+0000 7f6b33772700 1 -- 172.21.15.64:0/2439506917 wait complete. 2024-05-08T22:07:27.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:27 smithi064 ceph-mon[42500]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 660 op/s 2024-05-08T22:07:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:27 smithi160 ceph-mon[46962]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 660 op/s 2024-05-08T22:07:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:27 smithi186 ceph-mon[46619]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 660 op/s 2024-05-08T22:07:29.970 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:29 smithi064 ceph-mon[42500]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 596 op/s 2024-05-08T22:07:30.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:29 smithi160 ceph-mon[46962]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 596 op/s 2024-05-08T22:07:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:29 smithi186 ceph-mon[46619]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 596 op/s 2024-05-08T22:07:31.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:30 smithi160 ceph-mon[46962]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-08T22:07:31.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:30 smithi186 ceph-mon[46619]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-08T22:07:31.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:30 smithi064 ceph-mon[42500]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-08T22:07:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 760.0 s, 1468.2 tps, lat 67.516 ms stddev 81.569 2024-05-08T22:07:33.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:33 smithi064 ceph-mon[42500]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 643 op/s 2024-05-08T22:07:34.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:33 smithi160 ceph-mon[46962]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 643 op/s 2024-05-08T22:07:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:33 smithi186 ceph-mon[46619]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 643 op/s 2024-05-08T22:07:34.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:34 smithi064 ceph-mon[42500]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 616 op/s 2024-05-08T22:07:35.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:34 smithi160 ceph-mon[46962]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 616 op/s 2024-05-08T22:07:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:34 smithi186 ceph-mon[46619]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 616 op/s 2024-05-08T22:07:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 765.0 s, 1523.7 tps, lat 65.470 ms stddev 75.708 2024-05-08T22:07:37.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:37 smithi064 ceph-mon[42500]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 714 op/s 2024-05-08T22:07:38.055 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:37 smithi160 ceph-mon[46962]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 714 op/s 2024-05-08T22:07:38.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:37 smithi186 ceph-mon[46619]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 714 op/s 2024-05-08T22:07:39.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:38 smithi160 ceph-mon[46962]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 MiB/s wr, 655 op/s 2024-05-08T22:07:39.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:38 smithi186 ceph-mon[46619]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 MiB/s wr, 655 op/s 2024-05-08T22:07:39.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:38 smithi064 ceph-mon[42500]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 MiB/s wr, 655 op/s 2024-05-08T22:07:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:39 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:40.147 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:39 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:39 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.835+0000 7fbde117c700 1 -- 172.21.15.64:0/2383249582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 msgr2=0x7fbddc100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.835+0000 7fbde117c700 1 --2- 172.21.15.64:0/2383249582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 0x7fbddc100220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00fb20 tx=0x7fbdcc00fe30 comp rx=0 tx=0).stop 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.836+0000 7fbde117c700 1 -- 172.21.15.64:0/2383249582 shutdown_connections 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.836+0000 7fbde117c700 1 --2- 172.21.15.64:0/2383249582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 0x7fbddc100220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.836+0000 7fbde117c700 1 --2- 172.21.15.64:0/2383249582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.836+0000 7fbde117c700 1 --2- 172.21.15.64:0/2383249582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc100ff0 0x7fbddc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.836+0000 7fbde117c700 1 -- 172.21.15.64:0/2383249582 >> 172.21.15.64:0/2383249582 conn(0x7fbddc0fb390 msgr2=0x7fbddc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.836+0000 7fbde117c700 1 -- 172.21.15.64:0/2383249582 shutdown_connections 2024-05-08T22:07:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.837+0000 7fbde117c700 1 -- 172.21.15.64:0/2383249582 wait complete. 2024-05-08T22:07:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.837+0000 7fbde117c700 1 Processor -- start 2024-05-08T22:07:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 -- start start 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 0x7fbddc194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc100ff0 0x7fbddc195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc19c830 con 0x7fbddc100ff0 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc19c9a0 con 0x7fbddc0ffe00 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbde117c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc19cb60 con 0x7fbddc101990 2024-05-08T22:07:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbddb59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbddb59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:45980/0 (socket says 172.21.15.64:45980) 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbddb59e700 1 -- 172.21.15.64:0/3698471599 learned_addr learned my addr 172.21.15.64:0/3698471599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbddb59e700 1 -- 172.21.15.64:0/3698471599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 msgr2=0x7fbddc194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbddb59e700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 0x7fbddc194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.838+0000 7fbddb59e700 1 -- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc100ff0 msgr2=0x7fbddc195280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbddb59e700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc100ff0 0x7fbddc195280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbddb59e700 1 -- 172.21.15.64:0/3698471599 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc00f750 con 0x7fbddc101990 2024-05-08T22:07:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbddb59e700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc19c260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400bf00 tx=0x7fbdd40086b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbdc3fff700 1 -- 172.21.15.64:0/3698471599 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4010070 con 0x7fbddc101990 2024-05-08T22:07:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbdc3fff700 1 -- 172.21.15.64:0/3698471599 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd4008cc0 con 0x7fbddc101990 2024-05-08T22:07:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbdc3fff700 1 -- 172.21.15.64:0/3698471599 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4016d70 con 0x7fbddc101990 2024-05-08T22:07:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc19ce40 con 0x7fbddc101990 2024-05-08T22:07:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.840+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc19d270 con 0x7fbddc101990 2024-05-08T22:07:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.843+0000 7fbdc3fff700 1 -- 172.21.15.64:0/3698471599 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbdd4014070 con 0x7fbddc101990 2024-05-08T22:07:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.843+0000 7fbdc3fff700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc40a12f0 0x7fbdc40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.843+0000 7fbdc3fff700 1 -- 172.21.15.64:0/3698471599 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbdd40cc020 con 0x7fbddc101990 2024-05-08T22:07:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.843+0000 7fbddad9d700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc40a12f0 0x7fbdc40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.844+0000 7fbddad9d700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc40a12f0 0x7fbdc40a37b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006010 tx=0x7fbdcc003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.844+0000 7fbded336b80 1 Processor -- start 2024-05-08T22:07:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.844+0000 7fbded336b80 1 -- start start 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbded336b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f0996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbded336b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f09a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbded336b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbded336b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4efe80f0 con 0x556e4f099290 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbded336b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4efe5ab0 con 0x556e4f099bf0 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbded336b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4efe13a0 con 0x556e4f09aae0 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f09a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f09a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:51922/0 (socket says 172.21.15.64:51922) 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 -- 172.21.15.64:0/860536093 learned_addr learned my addr 172.21.15.64:0/860536093 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbddad9d700 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f0996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbddb59e700 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 -- 172.21.15.64:0/860536093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 msgr2=0x556e4f09af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 -- 172.21.15.64:0/860536093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 msgr2=0x556e4f0996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f0996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.845+0000 7fbdda59c700 1 -- 172.21.15.64:0/860536093 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e4ee62350 con 0x556e4f099bf0 2024-05-08T22:07:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.848+0000 7fbdda59c700 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f09a500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009830 tx=0x7fbdd0009b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14385a80aed01b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.848+0000 7fbdc17fa700 1 -- 172.21.15.64:0/860536093 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0011830 con 0x556e4f099bf0 2024-05-08T22:07:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.848+0000 7fbdc17fa700 1 -- 172.21.15.64:0/860536093 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd0011e50 con 0x556e4f099bf0 2024-05-08T22:07:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.848+0000 7fbdc17fa700 1 -- 172.21.15.64:0/860536093 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000b360 con 0x556e4f099bf0 2024-05-08T22:07:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- 172.21.15.64:0/860536093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 msgr2=0x556e4f09a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f09a500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009830 tx=0x7fbdd0009b40 comp rx=0 tx=0).stop 2024-05-08T22:07:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- 172.21.15.64:0/860536093 shutdown_connections 2024-05-08T22:07:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f0996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- 172.21.15.64:0/860536093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f09a500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- 172.21.15.64:0/860536093 >> 172.21.15.64:0/860536093 conn(0x556e4efef1b0 msgr2=0x556e4f08d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- 172.21.15.64:0/860536093 shutdown_connections 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- 172.21.15.64:0/860536093 wait complete. 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- start start 2024-05-08T22:07:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 Processor -- start 2024-05-08T22:07:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f091e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f092360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f09aae0 0x556e4f097340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4ef7d500 con 0x556e4f099290 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4ef7d1a0 con 0x556e4f09aae0 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4ee64640 con 0x556e4f099bf0 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbded336b80 10 client.0 _ll_register_callbacks cb 0x556e4ef86ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbdda59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f092360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.852+0000 7fbdda59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f092360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:46000/0 (socket says 172.21.15.64:46000) 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 -- 172.21.15.64:0/3269506969 learned_addr learned my addr 172.21.15.64:0/3269506969 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 -- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f09aae0 msgr2=0x556e4f097340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f09aae0 0x556e4f097340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 -- 172.21.15.64:0/3269506969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 msgr2=0x556e4f091e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f091e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 -- 172.21.15.64:0/3269506969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd0009460 con 0x556e4f099bf0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdda59c700 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f092360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009f10 tx=0x7fbdd0010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=153ddb6a4b2943ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdbeff5700 1 -- 172.21.15.64:0/3269506969 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0011830 con 0x556e4f099bf0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbdbeff5700 1 -- 172.21.15.64:0/3269506969 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbdd0011990 con 0x556e4f099bf0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.853+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e4ef3a210 con 0x556e4f099bf0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbdbeff5700 1 -- 172.21.15.64:0/3269506969 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd0010ef0 con 0x556e4f099bf0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 10 client.25750 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556e4ee0eb00 con 0x556e4f099bf0 2024-05-08T22:07:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbdbeff5700 1 -- 172.21.15.64:0/3269506969 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e4ef3a210 con 0x556e4f099bf0 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbdbeff5700 1 -- 172.21.15.64:0/3269506969 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbdd0022030 con 0x556e4f099bf0 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 10 client.25750 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 20 client.25750 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0a6680 0x556e4f0a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 4 client.25750 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.854+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e4f0607b0 con 0x556e4f0a6680 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.855+0000 7fbddad9d700 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0a6680 0x556e4f0a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.856+0000 7fbdbeff5700 10 client.25750 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.856+0000 7fbddad9d700 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0a6680 0x556e4f0a8ac0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a73fcb77fe3d3f server_cookie=1f4a7ac6a85bb446 in_seq=0 out_seq=0 2024-05-08T22:07:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.856+0000 7fbdbeff5700 1 -- 172.21.15.64:0/3269506969 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e4f0607b0 con 0x556e4f0a6680 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.856+0000 7fbdbeff5700 10 client.25750 handle_command_reply: tid=0 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 client.25750 shutdown 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 msgr2=0x556e4f092360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f092360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009f10 tx=0x7fbdd0010660 comp rx=0 tx=0).stop 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 shutdown_connections 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f09aae0 0x556e4f097340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0a6680 0x556e4f0a8ac0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f091e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 --2- 172.21.15.64:0/3269506969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f092360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 >> 172.21.15.64:0/3269506969 conn(0x556e4efef1b0 msgr2=0x556e4f08b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbdbeff5700 0 client.25750 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 shutdown_connections 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 1 -- 172.21.15.64:0/3269506969 wait complete. 2024-05-08T22:07:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.857+0000 7fbded336b80 20 client.25750 trim_cache size 0 max 16384 2024-05-08T22:07:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.903+0000 7fbded336b80 1 Processor -- start 2024-05-08T22:07:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 -- start start 2024-05-08T22:07:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f099670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f097300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4f092fc0 con 0x556e4f099290 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4f0a0d00 con 0x556e4f099bf0 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbded336b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4f09e610 con 0x556e4f09aae0 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:46002/0 (socket says 172.21.15.64:46002) 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 -- 172.21.15.64:0/1768833648 learned_addr learned my addr 172.21.15.64:0/1768833648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddad9d700 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 -- 172.21.15.64:0/1768833648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 msgr2=0x556e4f097300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f097300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 -- 172.21.15.64:0/1768833648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 msgr2=0x556e4f099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f099670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 -- 172.21.15.64:0/1768833648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e4efa7500 con 0x556e4f09aae0 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbddb59e700 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09aec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4008ab0 tx=0x7fbdd40c9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c7fba4e79435b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbdc1ffb700 1 -- 172.21.15.64:0/1768833648 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4099420 con 0x556e4f09aae0 2024-05-08T22:07:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbdc1ffb700 1 -- 172.21.15.64:0/1768833648 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd409d250 con 0x556e4f09aae0 2024-05-08T22:07:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.907+0000 7fbdc1ffb700 1 -- 172.21.15.64:0/1768833648 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4098b70 con 0x556e4f09aae0 2024-05-08T22:07:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 -- 172.21.15.64:0/1768833648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 msgr2=0x556e4f09aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09aec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4008ab0 tx=0x7fbdd40c9f90 comp rx=0 tx=0).stop 2024-05-08T22:07:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 -- 172.21.15.64:0/1768833648 shutdown_connections 2024-05-08T22:07:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f09aae0 0x556e4f09aec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f099290 0x556e4f099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 --2- 172.21.15.64:0/1768833648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099bf0 0x556e4f097300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.912+0000 7fbded336b80 1 -- 172.21.15.64:0/1768833648 >> 172.21.15.64:0/1768833648 conn(0x556e4efef1b0 msgr2=0x556e4f091330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.916+0000 7fbded336b80 1 -- 172.21.15.64:0/1768833648 shutdown_connections 2024-05-08T22:07:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.916+0000 7fbded336b80 1 -- 172.21.15.64:0/1768833648 wait complete. 2024-05-08T22:07:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.916+0000 7fbded336b80 1 -- start start 2024-05-08T22:07:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.916+0000 7fbded336b80 1 Processor -- start 2024-05-08T22:07:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099290 0x556e4f0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f09aae0 0x556e4f0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4f0b7190 con 0x556e4f09aae0 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4f0fb620 con 0x556e4f099290 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e4f0fb760 con 0x556e4f099bf0 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbded336b80 10 client.0 _ll_register_callbacks cb 0x556e4ef86ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbdda59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbddad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099290 0x556e4f0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbdda59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f0b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:46016/0 (socket says 172.21.15.64:46016) 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbdda59c700 1 -- 172.21.15.64:0/370174981 learned_addr learned my addr 172.21.15.64:0/370174981 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.917+0000 7fbddb59e700 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f09aae0 0x556e4f0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdda59c700 1 -- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099290 msgr2=0x556e4f0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdda59c700 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099290 0x556e4f0b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdda59c700 1 -- 172.21.15.64:0/370174981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f09aae0 msgr2=0x556e4f0b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdda59c700 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f09aae0 0x556e4f0b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdda59c700 1 -- 172.21.15.64:0/370174981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc19d270 con 0x556e4f099bf0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdda59c700 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f0b5ee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0010850 tx=0x7fbdd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73908e8c60bd5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdbeff5700 1 -- 172.21.15.64:0/370174981 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0006ac0 con 0x556e4f099bf0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdbeff5700 1 -- 172.21.15.64:0/370174981 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbdd0000e40 con 0x556e4f099bf0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdd0011e50 con 0x556e4f099bf0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.918+0000 7fbdbeff5700 1 -- 172.21.15.64:0/370174981 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd0024700 con 0x556e4f099bf0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 10 client.25756 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556e4ee66c70 con 0x556e4f099bf0 2024-05-08T22:07:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbdbeff5700 1 -- 172.21.15.64:0/370174981 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd0011e50 con 0x556e4f099bf0 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbdbeff5700 1 -- 172.21.15.64:0/370174981 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbdd0022040 con 0x556e4f099bf0 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 20 client.25756 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0fcea0 0x556e4f0ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 4 client.25756 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e4efd9e60 con 0x556e4f0fcea0 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.919+0000 7fbddad9d700 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0fcea0 0x556e4f0ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.920+0000 7fbdbeff5700 10 client.25756 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.920+0000 7fbddad9d700 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0fcea0 0x556e4f0ff2c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b2f979ef65e53a server_cookie=f0722e60eec0d3d0 in_seq=0 out_seq=0 2024-05-08T22:07:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbdbeff5700 1 -- 172.21.15.64:0/370174981 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e4efd9e60 con 0x556e4f0fcea0 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbdbeff5700 10 client.25756 handle_command_reply: tid=0 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 client.25756 shutdown 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 msgr2=0x556e4f0b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f0b5ee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0010850 tx=0x7fbdd0009260 comp rx=0 tx=0).stop 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 shutdown_connections 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e4f099290 0x556e4f0b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x556e4f0fcea0 0x556e4f0ff2c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e4f09aae0 0x556e4f0b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbdbeff5700 0 client.25756 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 --2- 172.21.15.64:0/370174981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e4f099bf0 0x556e4f0b5ee0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 >> 172.21.15.64:0/370174981 conn(0x556e4efef1b0 msgr2=0x556e4f0ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 shutdown_connections 2024-05-08T22:07:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.927+0000 7fbded336b80 1 -- 172.21.15.64:0/370174981 wait complete. 2024-05-08T22:07:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbded336b80 20 client.25756 trim_cache size 0 max 16384 2024-05-08T22:07:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:07:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:07:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:07:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:07:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc40a12f0 msgr2=0x7fbdc40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc40a12f0 0x7fbdc40a37b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006010 tx=0x7fbdcc003680 comp rx=0 tx=0).stop 2024-05-08T22:07:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 msgr2=0x7fbddc19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc19c260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400bf00 tx=0x7fbdd40086b0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 shutdown_connections 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fbdc40a12f0 0x7fbdc40a37b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc0ffe00 0x7fbddc194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbddc101990 0x7fbddc19c260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 --2- 172.21.15.64:0/3698471599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc100ff0 0x7fbddc195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 >> 172.21.15.64:0/3698471599 conn(0x7fbddc0fb390 msgr2=0x7fbddc1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 shutdown_connections 2024-05-08T22:07:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:40.929+0000 7fbde117c700 1 -- 172.21.15.64:0/3698471599 wait complete. 2024-05-08T22:07:41.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:40 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:41.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:40 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:41.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:40 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:07:41.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:40 smithi160 ceph-mon[46962]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 742 op/s 2024-05-08T22:07:41.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:40 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:41.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:40 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:41.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:40 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:07:41.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:40 smithi186 ceph-mon[46619]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 742 op/s 2024-05-08T22:07:41.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:41.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:41.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:07:41.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:40 smithi064 ceph-mon[42500]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 742 op/s 2024-05-08T22:07:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 770.0 s, 1483.9 tps, lat 68.141 ms stddev 82.142 2024-05-08T22:07:43.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:43 smithi064 ceph-mon[42500]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 694 op/s 2024-05-08T22:07:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:43 smithi160 ceph-mon[46962]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 694 op/s 2024-05-08T22:07:44.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:43 smithi186 ceph-mon[46619]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 694 op/s 2024-05-08T22:07:45.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:45 smithi064 ceph-mon[42500]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 641 op/s 2024-05-08T22:07:46.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:45 smithi160 ceph-mon[46962]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 641 op/s 2024-05-08T22:07:46.151 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:45 smithi186 ceph-mon[46619]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 641 op/s 2024-05-08T22:07:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 775.0 s, 1248.4 tps, lat 79.694 ms stddev 86.805 2024-05-08T22:07:47.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:47 smithi064 ceph-mon[42500]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-05-08T22:07:48.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:47 smithi160 ceph-mon[46962]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-05-08T22:07:48.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:47 smithi186 ceph-mon[46619]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-05-08T22:07:49.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:49 smithi064 ceph-mon[42500]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 674 op/s 2024-05-08T22:07:50.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:49 smithi160 ceph-mon[46962]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 674 op/s 2024-05-08T22:07:50.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:49 smithi186 ceph-mon[46619]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 674 op/s 2024-05-08T22:07:51.058 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:50 smithi160 ceph-mon[46962]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 798 op/s 2024-05-08T22:07:51.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:50 smithi186 ceph-mon[46619]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 798 op/s 2024-05-08T22:07:51.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:50 smithi064 ceph-mon[42500]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 798 op/s 2024-05-08T22:07:51.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 780.0 s, 1499.0 tps, lat 67.181 ms stddev 79.096 2024-05-08T22:07:53.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:53 smithi064 ceph-mon[42500]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T22:07:54.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:53 smithi160 ceph-mon[46962]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T22:07:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.112+0000 7f1e8c179700 1 -- 172.21.15.64:0/2861686007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e841090d0 msgr2=0x7f1e8410f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.112+0000 7f1e8c179700 1 --2- 172.21.15.64:0/2861686007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e841090d0 0x7f1e8410f980 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009fc0 tx=0x7f1e7c00b330 comp rx=0 tx=0).stop 2024-05-08T22:07:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.115+0000 7f1e8c179700 1 -- 172.21.15.64:0/2861686007 shutdown_connections 2024-05-08T22:07:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.115+0000 7f1e8c179700 1 --2- 172.21.15.64:0/2861686007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e841090d0 0x7f1e8410f980 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.115+0000 7f1e8c179700 1 --2- 172.21.15.64:0/2861686007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e84107540 0x7f1e84107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.115+0000 7f1e8c179700 1 --2- 172.21.15.64:0/2861686007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84108730 0x7f1e84108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.115+0000 7f1e8c179700 1 -- 172.21.15.64:0/2861686007 >> 172.21.15.64:0/2861686007 conn(0x7f1e840764d0 msgr2=0x7f1e84078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.118+0000 7f1e8c179700 1 -- 172.21.15.64:0/2861686007 shutdown_connections 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.120+0000 7f1e8c179700 1 -- 172.21.15.64:0/2861686007 wait complete. 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.121+0000 7f1e8c179700 1 Processor -- start 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.121+0000 7f1e8c179700 1 -- start start 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.121+0000 7f1e8c179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 0x7f1e84195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.121+0000 7f1e8c179700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e84108730 0x7f1e84195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.122+0000 7f1e8c179700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e841090d0 0x7f1e8419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.122+0000 7f1e8c179700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8419c820 con 0x7f1e84107540 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.122+0000 7f1e8c179700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84079820 con 0x7f1e84108730 2024-05-08T22:07:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.122+0000 7f1e8c179700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840799e0 con 0x7f1e841090d0 2024-05-08T22:07:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.127+0000 7f1e89f15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 0x7f1e84195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.127+0000 7f1e89f15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 0x7f1e84195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53216/0 (socket says 172.21.15.64:53216) 2024-05-08T22:07:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.127+0000 7f1e89f15700 1 -- 172.21.15.64:0/1684056441 learned_addr learned my addr 172.21.15.64:0/1684056441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.127+0000 7f1e8a716700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e841090d0 0x7f1e8419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.127+0000 7f1e89714700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e84108730 0x7f1e84195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.128+0000 7f1e89f15700 1 -- 172.21.15.64:0/1684056441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e841090d0 msgr2=0x7f1e8419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.128+0000 7f1e89f15700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e841090d0 0x7f1e8419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.128+0000 7f1e89f15700 1 -- 172.21.15.64:0/1684056441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e84108730 msgr2=0x7f1e84195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.128+0000 7f1e89f15700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e84108730 0x7f1e84195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.128+0000 7f1e89f15700 1 -- 172.21.15.64:0/1684056441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c009c40 con 0x7f1e84107540 2024-05-08T22:07:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.129+0000 7f1e89f15700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 0x7f1e84195450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000e970 tx=0x7f1e8000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.129+0000 7f1e7affd700 1 -- 172.21.15.64:0/1684056441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80009790 con 0x7f1e84107540 2024-05-08T22:07:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.129+0000 7f1e7affd700 1 -- 172.21.15.64:0/1684056441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8000eec0 con 0x7f1e84107540 2024-05-08T22:07:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.129+0000 7f1e7affd700 1 -- 172.21.15.64:0/1684056441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80003cf0 con 0x7f1e84107540 2024-05-08T22:07:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.130+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e84079cc0 con 0x7f1e84107540 2024-05-08T22:07:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.130+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8407a180 con 0x7f1e84107540 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 Processor -- start 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 -- start start 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269aa80 0x55b0a269aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269b3e0 0x55b0a269bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a269c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a25f20f0 con 0x55b0a269b3e0 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a25efab0 con 0x55b0a269aa80 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.135+0000 7f1e98333b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a25eb3a0 con 0x55b0a269c2d0 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.136+0000 7f1e8a716700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269aa80 0x55b0a269aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.137+0000 7f1e89f15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269b3e0 0x55b0a269bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.137+0000 7f1e89f15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269b3e0 0x55b0a269bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53230/0 (socket says 172.21.15.64:53230) 2024-05-08T22:07:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.137+0000 7f1e89f15700 1 -- 172.21.15.64:0/2030836763 learned_addr learned my addr 172.21.15.64:0/2030836763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e89714700 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a269c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e8a716700 1 -- 172.21.15.64:0/2030836763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 msgr2=0x55b0a269c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e8a716700 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a269c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e8a716700 1 -- 172.21.15.64:0/2030836763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269b3e0 msgr2=0x55b0a269bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e8a716700 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269b3e0 0x55b0a269bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e8a716700 1 -- 172.21.15.64:0/2030836763 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0a246c350 con 0x55b0a269aa80 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.140+0000 7f1e8a716700 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269aa80 0x55b0a269aea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c007a30 tx=0x7f1e7c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7320984ef634d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.141+0000 7f1e6ffff700 1 -- 172.21.15.64:0/2030836763 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00e050 con 0x55b0a269aa80 2024-05-08T22:07:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.141+0000 7f1e6ffff700 1 -- 172.21.15.64:0/2030836763 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7c0075e0 con 0x55b0a269aa80 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.141+0000 7f1e6ffff700 1 -- 172.21.15.64:0/2030836763 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00bd70 con 0x55b0a269aa80 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 -- 172.21.15.64:0/2030836763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269aa80 msgr2=0x55b0a269aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269aa80 0x55b0a269aea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c007a30 tx=0x7f1e7c009ba0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 -- 172.21.15.64:0/2030836763 shutdown_connections 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269aa80 0x55b0a269aea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a269c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2030836763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269b3e0 0x55b0a269bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.142+0000 7f1e98333b80 1 -- 172.21.15.64:0/2030836763 >> 172.21.15.64:0/2030836763 conn(0x55b0a25f91b0 msgr2=0x55b0a2697440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:54.145 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:53 smithi186 ceph-mon[46619]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T22:07:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.144+0000 7f1e7affd700 1 -- 172.21.15.64:0/1684056441 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1e8000ccd0 con 0x7f1e84107540 2024-05-08T22:07:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.144+0000 7f1e98333b80 1 -- 172.21.15.64:0/2030836763 shutdown_connections 2024-05-08T22:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.144+0000 7f1e98333b80 1 -- 172.21.15.64:0/2030836763 wait complete. 2024-05-08T22:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.145+0000 7f1e98333b80 1 -- start start 2024-05-08T22:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.145+0000 7f1e7affd700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1e700a10b0 0x7f1e700a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.145+0000 7f1e7affd700 1 -- 172.21.15.64:0/1684056441 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1e80014070 con 0x7f1e84107540 2024-05-08T22:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.145+0000 7f1e8a716700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1e700a10b0 0x7f1e700a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 Processor -- start 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e8a716700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1e700a10b0 0x7f1e700a3570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c006130 tx=0x7f1e7c00b330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269c2d0 0x55b0a26c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 0x55b0a26ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a2587500 con 0x55b0a269aa80 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a25871a0 con 0x55b0a269c2d0 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a246e640 con 0x55b0a26cbf40 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e98333b80 10 client.0 _ll_register_callbacks cb 0x55b0a2590ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e8a716700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 0x55b0a26ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e8a716700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 0x55b0a26ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48618/0 (socket says 172.21.15.64:48618) 2024-05-08T22:07:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.146+0000 7f1e8a716700 1 -- 172.21.15.64:0/2883847375 learned_addr learned my addr 172.21.15.64:0/2883847375 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.147+0000 7f1e89714700 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269c2d0 0x55b0a26c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.147+0000 7f1e8a716700 1 -- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269c2d0 msgr2=0x55b0a26c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.147+0000 7f1e8a716700 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269c2d0 0x55b0a26c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.147+0000 7f1e8a716700 1 -- 172.21.15.64:0/2883847375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 msgr2=0x55b0a26c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.147+0000 7f1e8a716700 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.147+0000 7f1e8a716700 1 -- 172.21.15.64:0/2883847375 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e841102d0 con 0x55b0a26cbf40 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.148+0000 7f1e8a716700 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 0x55b0a26ce330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c004e80 tx=0x7f1e7c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5d2e358198808d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.148+0000 7f1e6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:07:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.148+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2883847375 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c007ea0 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.148+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2883847375 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1e7c01b430 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.149+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0a2544210 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2883847375 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e7c03a680 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 10 client.25768 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55b0a2418b00 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2883847375 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7c041030 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2883847375 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1e7c01ba70 con 0x55b0a26cbf40 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 10 client.25768 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 20 client.25768 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a26d26c0 0x55b0a26d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 4 client.25768 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.150+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0a266a7b0 con 0x55b0a26d26c0 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.151+0000 7f1e89f15700 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a26d26c0 0x55b0a26d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.151+0000 7f1e89f15700 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a26d26c0 0x55b0a26d4ab0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3ebc53c481cae8 server_cookie=18ea5d9f21a809d1 in_seq=0 out_seq=0 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.151+0000 7f1e6d7fa700 10 client.25768 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.152+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2883847375 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b0a266a7b0 con 0x55b0a26d26c0 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.152+0000 7f1e6d7fa700 10 client.25768 handle_command_reply: tid=0 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.152+0000 7f1e98333b80 1 client.25768 shutdown 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.152+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 msgr2=0x55b0a26ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.152+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 0x55b0a26ce330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c004e80 tx=0x7f1e7c009510 comp rx=0 tx=0).stop 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 shutdown_connections 2024-05-08T22:07:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a26d26c0 0x55b0a26d4ab0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a26cbf40 0x55b0a26ce330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2883847375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a269c2d0 0x55b0a26c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 >> 172.21.15.64:0/2883847375 conn(0x55b0a25f91b0 msgr2=0x55b0a2695860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e6d7fa700 0 client.25768 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 shutdown_connections 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 1 -- 172.21.15.64:0/2883847375 wait complete. 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.154+0000 7f1e98333b80 20 client.25768 trim_cache size 0 max 16384 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 Processor -- start 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 -- start start 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269aa80 0x55b0a26a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 0x55b0a269c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a269bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a2690330 con 0x55b0a269c2d0 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a26929b0 con 0x55b0a26cbf40 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e98333b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a2690570 con 0x55b0a269aa80 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 0x55b0a269c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89f15700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269aa80 0x55b0a26a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 0x55b0a269c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53262/0 (socket says 172.21.15.64:53262) 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e8a716700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a269bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 -- 172.21.15.64:0/1415059786 learned_addr learned my addr 172.21.15.64:0/1415059786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 -- 172.21.15.64:0/1415059786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269aa80 msgr2=0x55b0a26a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269aa80 0x55b0a26a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 -- 172.21.15.64:0/1415059786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 msgr2=0x55b0a269bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.173+0000 7f1e89714700 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a269bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.174+0000 7f1e89714700 1 -- 172.21.15.64:0/1415059786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0a25b1500 con 0x55b0a269c2d0 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.174+0000 7f1e89714700 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 0x55b0a269c6b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1e740046d0 tx=0x7f1e7401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b504fe91e1c0414 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.174+0000 7f1e78ff9700 1 -- 172.21.15.64:0/1415059786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74028070 con 0x55b0a269c2d0 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.174+0000 7f1e78ff9700 1 -- 172.21.15.64:0/1415059786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e74003d50 con 0x55b0a269c2d0 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.174+0000 7f1e78ff9700 1 -- 172.21.15.64:0/1415059786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74022bc0 con 0x55b0a269c2d0 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.175+0000 7f1e98333b80 1 -- 172.21.15.64:0/1415059786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 msgr2=0x55b0a269c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.175+0000 7f1e98333b80 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 0x55b0a269c6b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1e740046d0 tx=0x7f1e7401ca00 comp rx=0 tx=0).stop 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 -- 172.21.15.64:0/1415059786 shutdown_connections 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a269bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269aa80 0x55b0a26a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 --2- 172.21.15.64:0/1415059786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269c2d0 0x55b0a269c6b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 -- 172.21.15.64:0/1415059786 >> 172.21.15.64:0/1415059786 conn(0x55b0a25f91b0 msgr2=0x55b0a26c5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 -- 172.21.15.64:0/1415059786 shutdown_connections 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.177+0000 7f1e98333b80 1 -- 172.21.15.64:0/1415059786 wait complete. 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 -- start start 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 Processor -- start 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26e1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a26e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a26e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a26e3270 con 0x55b0a269aa80 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a2727610 con 0x55b0a26cbf40 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a27277d0 con 0x55b0a269c2d0 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e98333b80 10 client.0 _ll_register_callbacks cb 0x55b0a2590ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e89f15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e89714700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a26e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e8a716700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a26e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e89714700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a26e1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48626/0 (socket says 172.21.15.64:48626) 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.178+0000 7f1e89714700 1 -- 172.21.15.64:0/2080410893 learned_addr learned my addr 172.21.15.64:0/2080410893 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89714700 1 -- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 msgr2=0x55b0a26e21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89714700 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a26e21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89714700 1 -- 172.21.15.64:0/2080410893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 msgr2=0x55b0a26e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89714700 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89714700 1 -- 172.21.15.64:0/2080410893 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e74004350 con 0x55b0a269c2d0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89f15700 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e89714700 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a26e1c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003af0 tx=0x7f1e74005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc3960fdc576502 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2080410893 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74028070 con 0x55b0a269c2d0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e7c0075e0 con 0x55b0a269c2d0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2080410893 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1e74022cd0 con 0x55b0a269c2d0 2024-05-08T22:07:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2080410893 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e740226e0 con 0x55b0a269c2d0 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e98333b80 10 client.25774 fetch_fsmap learned FSMap version 47 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.179+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55b0a2470c70 con 0x55b0a269c2d0 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2080410893 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7c0075e0 con 0x55b0a269c2d0 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2080410893 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1e74004850 con 0x55b0a269c2d0 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e98333b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e98333b80 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e98333b80 20 client.25774 populate_metadata read hostname 'smithi064' 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a27291b0 0x55b0a272b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e98333b80 4 client.25774 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b0a25e3e60 con 0x55b0a27291b0 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e89f15700 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a27291b0 0x55b0a272b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e6d7fa700 10 client.25774 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.180+0000 7f1e89f15700 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a27291b0 0x55b0a272b5d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba9feb79c7c37a1 server_cookie=55c4638c49b70ab0 in_seq=0 out_seq=0 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e6d7fa700 1 -- 172.21.15.64:0/2080410893 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b0a25e3e60 con 0x55b0a27291b0 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e6d7fa700 10 client.25774 handle_command_reply: tid=0 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 client.25774 shutdown 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 msgr2=0x55b0a26e1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a26e1c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003af0 tx=0x7f1e74005b00 comp rx=0 tx=0).stop 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 shutdown_connections 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b0a26cbf40 0x55b0a26e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55b0a27291b0 0x55b0a272b5d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0a269aa80 0x55b0a26e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 --2- 172.21.15.64:0/2080410893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0a269c2d0 0x55b0a26e1c70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e6d7fa700 0 client.25774 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.191+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 >> 172.21.15.64:0/2080410893 conn(0x55b0a25f91b0 msgr2=0x55b0a2697020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 shutdown_connections 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e98333b80 1 -- 172.21.15.64:0/2080410893 wait complete. 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e98333b80 20 client.25774 trim_cache size 0 max 16384 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:07:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:07:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:07:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:07:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1e700a10b0 msgr2=0x7f1e700a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1e700a10b0 0x7f1e700a3570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c006130 tx=0x7f1e7c00b330 comp rx=0 tx=0).stop 2024-05-08T22:07:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 msgr2=0x7f1e84195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 0x7f1e84195450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000e970 tx=0x7f1e8000ec80 comp rx=0 tx=0).stop 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 shutdown_connections 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1e700a10b0 0x7f1e700a3570 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e841090d0 0x7f1e8419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e84107540 0x7f1e84195450 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 --2- 172.21.15.64:0/1684056441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e84108730 0x7f1e84195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 >> 172.21.15.64:0/1684056441 conn(0x7f1e840764d0 msgr2=0x7f1e8406fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 shutdown_connections 2024-05-08T22:07:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:07:54.192+0000 7f1e8c179700 1 -- 172.21.15.64:0/1684056441 wait complete. 2024-05-08T22:07:55.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:55 smithi064 ceph-mon[42500]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-05-08T22:07:56.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:55 smithi160 ceph-mon[46962]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-05-08T22:07:56.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:55 smithi186 ceph-mon[46619]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-05-08T22:07:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 785.0 s, 1593.8 tps, lat 62.587 ms stddev 72.989 2024-05-08T22:07:57.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:57 smithi064 ceph-mon[42500]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 772 op/s 2024-05-08T22:07:58.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:57 smithi160 ceph-mon[46962]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 772 op/s 2024-05-08T22:07:58.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:57 smithi186 ceph-mon[46619]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 772 op/s 2024-05-08T22:07:59.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:07:59 smithi064 ceph-mon[42500]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 694 op/s 2024-05-08T22:08:00.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:07:59 smithi160 ceph-mon[46962]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 694 op/s 2024-05-08T22:08:00.158 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:07:59 smithi186 ceph-mon[46619]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 694 op/s 2024-05-08T22:08:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 790.0 s, 1300.0 tps, lat 75.920 ms stddev 81.139 2024-05-08T22:08:01.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:01 smithi064 ceph-mon[42500]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 739 op/s 2024-05-08T22:08:02.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:01 smithi160 ceph-mon[46962]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 739 op/s 2024-05-08T22:08:02.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:01 smithi186 ceph-mon[46619]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 739 op/s 2024-05-08T22:08:03.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:03 smithi064 ceph-mon[42500]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 677 op/s 2024-05-08T22:08:04.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:03 smithi160 ceph-mon[46962]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 677 op/s 2024-05-08T22:08:04.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:03 smithi186 ceph-mon[46619]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 677 op/s 2024-05-08T22:08:05.969 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:05 smithi064 ceph-mon[42500]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-08T22:08:06.052 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:05 smithi160 ceph-mon[46962]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-08T22:08:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:05 smithi186 ceph-mon[46619]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-08T22:08:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 795.0 s, 1046.3 tps, lat 90.116 ms stddev 107.923 2024-05-08T22:08:07.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:06 smithi186 ceph-mon[46619]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 742 op/s 2024-05-08T22:08:07.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:06 smithi064 ceph-mon[42500]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 742 op/s 2024-05-08T22:08:07.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:06 smithi160 ceph-mon[46962]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 742 op/s 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 -- 172.21.15.64:0/2544801725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 msgr2=0x7f521c0ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 --2- 172.21.15.64:0/2544801725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c0ff5e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5218009990 tx=0x7f5218009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 -- 172.21.15.64:0/2544801725 shutdown_connections 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 --2- 172.21.15.64:0/2544801725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f521c102bd0 0x7f521c109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 --2- 172.21.15.64:0/2544801725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c0ff5e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 --2- 172.21.15.64:0/2544801725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c102230 0x7f521c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 -- 172.21.15.64:0/2544801725 >> 172.21.15.64:0/2544801725 conn(0x7f521c0faaf0 msgr2=0x7f521c0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 -- 172.21.15.64:0/2544801725 shutdown_connections 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.344+0000 7f52236e7700 1 -- 172.21.15.64:0/2544801725 wait complete. 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 Processor -- start 2024-05-08T22:08:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 -- start start 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c102230 0x7f521c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f521c102bd0 0x7f521c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19e940 con 0x7f521c0ff1c0 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19ea80 con 0x7f521c102bd0 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f52236e7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19ec10 con 0x7f521c102230 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45652/0 (socket says 172.21.15.64:45652) 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 -- 172.21.15.64:0/15078591 learned_addr learned my addr 172.21.15.64:0/15078591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221c84700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f521c102bd0 0x7f521c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5220c82700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c102230 0x7f521c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 -- 172.21.15.64:0/15078591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c102230 msgr2=0x7f521c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c102230 0x7f521c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 -- 172.21.15.64:0/15078591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f521c102bd0 msgr2=0x7f521c19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f521c102bd0 0x7f521c19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 -- 172.21.15.64:0/15078591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52180095f0 con 0x7f521c0ff1c0 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.345+0000 7f5221483700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c196e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f52180098e0 tx=0x7f5218005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.346+0000 7f52127fc700 1 -- 172.21.15.64:0/15078591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521801d070 con 0x7f521c0ff1c0 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.346+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c19ee90 con 0x7f521c0ff1c0 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.346+0000 7f52127fc700 1 -- 172.21.15.64:0/15078591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5218022470 con 0x7f521c0ff1c0 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.346+0000 7f52127fc700 1 -- 172.21.15.64:0/15078591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218017650 con 0x7f521c0ff1c0 2024-05-08T22:08:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.346+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c19f200 con 0x7f521c0ff1c0 2024-05-08T22:08:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.349+0000 7f52127fc700 1 -- 172.21.15.64:0/15078591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f521800fb10 con 0x7f521c0ff1c0 2024-05-08T22:08:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.350+0000 7f52127fc700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52080a1120 0x7f52080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.350+0000 7f52127fc700 1 -- 172.21.15.64:0/15078591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f52180cdca0 con 0x7f521c0ff1c0 2024-05-08T22:08:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.350+0000 7f5220c82700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52080a1120 0x7f52080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.350+0000 7f5220c82700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52080a1120 0x7f52080a35e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f520c00ac50 tx=0x7f520c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.351+0000 7f522f8a1b80 1 Processor -- start 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.351+0000 7f522f8a1b80 1 -- start start 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.351+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.351+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a417f0 0x55e408a42100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.351+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a42b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4089940f0 con 0x55e408a40e90 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408991ab0 con 0x55e408a426e0 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40898d3a0 con 0x55e408a417f0 2024-05-08T22:08:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45668/0 (socket says 172.21.15.64:45668) 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 -- 172.21.15.64:0/3305280068 learned_addr learned my addr 172.21.15.64:0/3305280068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221c84700 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a42b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 -- 172.21.15.64:0/3305280068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a417f0 msgr2=0x55e408a42100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a417f0 0x55e408a42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 -- 172.21.15.64:0/3305280068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 msgr2=0x55e408a42b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a42b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 -- 172.21.15.64:0/3305280068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40880e350 con 0x55e408a40e90 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f5221483700 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a412b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f521800b680 tx=0x7f5218009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c9f9d2c53d0153 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f52077fe700 1 -- 172.21.15.64:0/3305280068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521809ba40 con 0x55e408a40e90 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f52077fe700 1 -- 172.21.15.64:0/3305280068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f521809b340 con 0x55e408a40e90 2024-05-08T22:08:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.352+0000 7f52077fe700 1 -- 172.21.15.64:0/3305280068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521809a830 con 0x55e408a40e90 2024-05-08T22:08:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.353+0000 7f522f8a1b80 1 -- 172.21.15.64:0/3305280068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 msgr2=0x55e408a412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.353+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a412b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f521800b680 tx=0x7f5218009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.363+0000 7f522f8a1b80 1 -- 172.21.15.64:0/3305280068 shutdown_connections 2024-05-08T22:08:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.363+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a42b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.363+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a412b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.363+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/3305280068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a417f0 0x55e408a42100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.363+0000 7f522f8a1b80 1 -- 172.21.15.64:0/3305280068 >> 172.21.15.64:0/3305280068 conn(0x55e40899b1b0 msgr2=0x55e408a39440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.364+0000 7f522f8a1b80 1 -- 172.21.15.64:0/3305280068 shutdown_connections 2024-05-08T22:08:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.366+0000 7f522f8a1b80 1 -- 172.21.15.64:0/3305280068 wait complete. 2024-05-08T22:08:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.366+0000 7f522f8a1b80 1 -- start start 2024-05-08T22:08:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.366+0000 7f522f8a1b80 1 Processor -- start 2024-05-08T22:08:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a46260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a426e0 0x55e408a477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408929500 con 0x55e408a40e90 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4089291a0 con 0x55e408a417f0 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408810640 con 0x55e408a426e0 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f522f8a1b80 10 client.0 _ll_register_callbacks cb 0x55e408932ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f5220c82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f5220c82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:59168/0 (socket says 172.21.15.64:59168) 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.367+0000 7f5220c82700 1 -- 172.21.15.64:0/2989042771 learned_addr learned my addr 172.21.15.64:0/2989042771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.368+0000 7f5220c82700 1 -- 172.21.15.64:0/2989042771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a426e0 msgr2=0x55e408a477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.368+0000 7f5220c82700 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a426e0 0x55e408a477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.368+0000 7f5220c82700 1 -- 172.21.15.64:0/2989042771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 msgr2=0x55e408a46260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.368+0000 7f5220c82700 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a46260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.368+0000 7f5220c82700 1 -- 172.21.15.64:0/2989042771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c19f200 con 0x55e408a417f0 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.372+0000 7f5220c82700 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a467a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f520c004660 tx=0x7f520c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8473d15e8da03439 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.372+0000 7f5204ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:08:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.377+0000 7f5204ff9700 1 -- 172.21.15.64:0/2989042771 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c010840 con 0x55e408a417f0 2024-05-08T22:08:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.377+0000 7f5204ff9700 1 -- 172.21.15.64:0/2989042771 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f520c0109a0 con 0x55e408a417f0 2024-05-08T22:08:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.377+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4088e6210 con 0x55e408a417f0 2024-05-08T22:08:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.384+0000 7f5204ff9700 1 -- 172.21.15.64:0/2989042771 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f520c00ae60 con 0x55e408a417f0 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.384+0000 7f522f8a1b80 10 client.25934 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.384+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55e4087bab00 con 0x55e408a417f0 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.384+0000 7f5204ff9700 1 -- 172.21.15.64:0/2989042771 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4088e6210 con 0x55e408a417f0 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.384+0000 7f5204ff9700 1 -- 172.21.15.64:0/2989042771 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f520c023020 con 0x55e408a417f0 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.385+0000 7f522f8a1b80 10 client.25934 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.385+0000 7f522f8a1b80 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.385+0000 7f522f8a1b80 20 client.25934 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.386+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408a52370 0x55e408a54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.386+0000 7f522f8a1b80 4 client.25934 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.386+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e408a0c7b0 con 0x55e408a52370 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.386+0000 7f5221483700 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408a52370 0x55e408a54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.386+0000 7f5221483700 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408a52370 0x55e408a54790 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12712257397d65e server_cookie=c207ff072dd3a48a in_seq=0 out_seq=0 2024-05-08T22:08:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.386+0000 7f5204ff9700 10 client.25934 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f5204ff9700 1 -- 172.21.15.64:0/2989042771 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e408a0c7b0 con 0x55e408a52370 2024-05-08T22:08:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f5204ff9700 10 client.25934 handle_command_reply: tid=0 2024-05-08T22:08:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 client.25934 shutdown 2024-05-08T22:08:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 msgr2=0x55e408a467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a467a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f520c004660 tx=0x7f520c004750 comp rx=0 tx=0).stop 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 shutdown_connections 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408a52370 0x55e408a54790 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a426e0 0x55e408a477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a40e90 0x55e408a46260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2989042771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a467a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 >> 172.21.15.64:0/2989042771 conn(0x55e40899b1b0 msgr2=0x55e408a37860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.393+0000 7f5204ff9700 0 client.25934 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.394+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 shutdown_connections 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.394+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2989042771 wait complete. 2024-05-08T22:08:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.394+0000 7f522f8a1b80 20 client.25934 trim_cache size 0 max 16384 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 Processor -- start 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 -- start start 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a42ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a417f0 0x55e408a476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408a0c7b0 con 0x55e408a417f0 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4088e6210 con 0x55e408a426e0 2024-05-08T22:08:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408a349b0 con 0x55e408a40e90 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.412+0000 7f5220c82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a417f0 0x55e408a476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5220c82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a417f0 0x55e408a476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45692/0 (socket says 172.21.15.64:45692) 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5220c82700 1 -- 172.21.15.64:0/2899848655 learned_addr learned my addr 172.21.15.64:0/2899848655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221483700 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a42ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 -- 172.21.15.64:0/2899848655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 msgr2=0x55e408a42ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a42ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 -- 172.21.15.64:0/2899848655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a417f0 msgr2=0x55e408a476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a417f0 0x55e408a476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 -- 172.21.15.64:0/2899848655 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e408953500 con 0x55e408a426e0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5221c84700 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a590f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55e408a41ec0 tx=0x7f52140148d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7d47e6560583ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5207fff700 1 -- 172.21.15.64:0/2899848655 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400e040 con 0x55e408a426e0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5207fff700 1 -- 172.21.15.64:0/2899848655 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f521400f040 con 0x55e408a426e0 2024-05-08T22:08:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.413+0000 7f5207fff700 1 -- 172.21.15.64:0/2899848655 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214007500 con 0x55e408a426e0 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2899848655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 msgr2=0x55e408a590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a590f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55e408a41ec0 tx=0x7f52140148d0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2899848655 shutdown_connections 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a426e0 0x55e408a590f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a42ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2899848655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a417f0 0x55e408a476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2899848655 >> 172.21.15.64:0/2899848655 conn(0x55e40899b1b0 msgr2=0x55e408a45770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.414+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2899848655 shutdown_connections 2024-05-08T22:08:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.415+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2899848655 wait complete. 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.415+0000 7f522f8a1b80 1 -- start start 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.415+0000 7f522f8a1b80 1 Processor -- start 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.415+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a3ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.415+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f522f8a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 0x55e408a4ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408a3f2f0 con 0x55e408a426e0 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408a4f190 con 0x55e408a417f0 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f522f8a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e408a4f350 con 0x55e408a40e90 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f522f8a1b80 10 client.0 _ll_register_callbacks cb 0x55e408932ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5221483700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a3ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5221c84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 0x55e408a4ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5221c84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 0x55e408a4ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45700/0 (socket says 172.21.15.64:45700) 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5221c84700 1 -- 172.21.15.64:0/2171734721 learned_addr learned my addr 172.21.15.64:0/2171734721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5221483700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a3ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:44072/0 (socket says 172.21.15.64:44072) 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 -- 172.21.15.64:0/2171734721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 msgr2=0x55e408a3ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a3ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 -- 172.21.15.64:0/2171734721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 msgr2=0x55e408a4ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 0x55e408a4ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 -- 172.21.15.64:0/2171734721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5214014360 con 0x55e408a417f0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5221c84700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 0x55e408a4ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5220c82700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a4e6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f520c00fa30 tx=0x7f520c025450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c040185660cf8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.416+0000 7f5204ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f5204ff9700 1 -- 172.21.15.64:0/2171734721 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c02f830 con 0x55e408a417f0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f5204ff9700 1 -- 172.21.15.64:0/2171734721 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f520c02f990 con 0x55e408a417f0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f521809b340 con 0x55e408a417f0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f5204ff9700 1 -- 172.21.15.64:0/2171734721 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f520c02fb50 con 0x55e408a417f0 2024-05-08T22:08:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 10 client.25940 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55e408812c70 con 0x55e408a417f0 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f5204ff9700 1 -- 172.21.15.64:0/2171734721 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f521809b340 con 0x55e408a417f0 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f5204ff9700 1 -- 172.21.15.64:0/2171734721 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f520c003730 con 0x55e408a417f0 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 10 client.25940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 20 client.25940 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408aca7e0 0x55e408accc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 4 client.25940 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e408985e60 con 0x55e408aca7e0 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.417+0000 7f5221483700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408aca7e0 0x55e408accc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.418+0000 7f5204ff9700 10 client.25940 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.418+0000 7f5221483700 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408aca7e0 0x55e408accc00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e458e6377f84aee8 server_cookie=3c53ade6c22e3af4 in_seq=0 out_seq=0 2024-05-08T22:08:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.442+0000 7f5204ff9700 1 -- 172.21.15.64:0/2171734721 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e408985e60 con 0x55e408aca7e0 2024-05-08T22:08:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.442+0000 7f5204ff9700 10 client.25940 handle_command_reply: tid=0 2024-05-08T22:08:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.442+0000 7f522f8a1b80 1 client.25940 shutdown 2024-05-08T22:08:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.442+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 msgr2=0x55e408a4e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.442+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a4e6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f520c00fa30 tx=0x7f520c025450 comp rx=0 tx=0).stop 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 shutdown_connections 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55e408aca7e0 0x55e408accc00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e408a426e0 0x55e408a4ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e408a40e90 0x55e408a3ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f5204ff9700 0 client.25940 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 --2- 172.21.15.64:0/2171734721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e408a417f0 0x55e408a4e6e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 >> 172.21.15.64:0/2171734721 conn(0x55e40899b1b0 msgr2=0x55e408a590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 shutdown_connections 2024-05-08T22:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.445+0000 7f522f8a1b80 1 -- 172.21.15.64:0/2171734721 wait complete. 2024-05-08T22:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.447+0000 7f522f8a1b80 20 client.25940 trim_cache size 0 max 16384 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.448+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52080a1120 msgr2=0x7f52080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.448+0000 7f52236e7700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52080a1120 0x7f52080a35e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f520c00ac50 tx=0x7f520c00a380 comp rx=0 tx=0).stop 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.448+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 msgr2=0x7f521c196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.448+0000 7f52236e7700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c196e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f52180098e0 tx=0x7f5218005720 comp rx=0 tx=0).stop 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 shutdown_connections 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f52080a1120 0x7f52080a35e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f521c102bd0 0x7f521c19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f521c0ff1c0 0x7f521c196e10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 --2- 172.21.15.64:0/15078591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c102230 0x7f521c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 >> 172.21.15.64:0/15078591 conn(0x7f521c0faaf0 msgr2=0x7f521c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 shutdown_connections 2024-05-08T22:08:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:07.449+0000 7f52236e7700 1 -- 172.21.15.64:0/15078591 wait complete. 2024-05-08T22:08:09.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:09 smithi064 ceph-mon[42500]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-05-08T22:08:10.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:09 smithi160 ceph-mon[46962]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-05-08T22:08:10.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:09 smithi186 ceph-mon[46619]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-05-08T22:08:11.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 800.0 s, 1069.5 tps, lat 100.111 ms stddev 200.512 2024-05-08T22:08:11.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:11 smithi064 ceph-mon[42500]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.68k op/s 2024-05-08T22:08:12.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:11 smithi160 ceph-mon[46962]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.68k op/s 2024-05-08T22:08:12.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:11 smithi186 ceph-mon[46619]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.68k op/s 2024-05-08T22:08:13.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:12 smithi160 ceph-mon[46962]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.96k op/s 2024-05-08T22:08:13.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:12 smithi186 ceph-mon[46619]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.96k op/s 2024-05-08T22:08:13.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:12 smithi064 ceph-mon[42500]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.96k op/s 2024-05-08T22:08:15.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:15 smithi064 ceph-mon[42500]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.95k op/s 2024-05-08T22:08:16.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:15 smithi160 ceph-mon[46962]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.95k op/s 2024-05-08T22:08:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:15 smithi186 ceph-mon[46619]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.95k op/s 2024-05-08T22:08:16.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 805.0 s, 1657.5 tps, lat 60.266 ms stddev 70.410 2024-05-08T22:08:17.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:16 smithi160 ceph-mon[46962]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 1.94k op/s 2024-05-08T22:08:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:16 smithi186 ceph-mon[46619]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 1.94k op/s 2024-05-08T22:08:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:16 smithi064 ceph-mon[42500]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 1.94k op/s 2024-05-08T22:08:19.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:19 smithi064 ceph-mon[42500]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 MiB/s wr, 1.86k op/s 2024-05-08T22:08:20.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:19 smithi160 ceph-mon[46962]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 MiB/s wr, 1.86k op/s 2024-05-08T22:08:20.156 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:19 smithi186 ceph-mon[46619]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 MiB/s wr, 1.86k op/s 2024-05-08T22:08:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 -- 172.21.15.64:0/741093381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4854104310 msgr2=0x7f4854106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 --2- 172.21.15.64:0/741093381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4854104310 0x7f4854106700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b0a0 tx=0x7f484c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 -- 172.21.15.64:0/741093381 shutdown_connections 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 --2- 172.21.15.64:0/741093381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f4854109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 --2- 172.21.15.64:0/741093381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48541019e0 0x7f4854103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 --2- 172.21.15.64:0/741093381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4854104310 0x7f4854106700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 -- 172.21.15.64:0/741093381 >> 172.21.15.64:0/741093381 conn(0x7f48540fb390 msgr2=0x7f48540fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 -- 172.21.15.64:0/741093381 shutdown_connections 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.624+0000 7f485b229700 1 -- 172.21.15.64:0/741093381 wait complete. 2024-05-08T22:08:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.625+0000 7f485b229700 1 Processor -- start 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.625+0000 7f485b229700 1 -- start start 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f485b229700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48541019e0 0x7f4854197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f485b229700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4854104310 0x7f4854197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f485b229700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f485419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f485b229700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485419eb30 con 0x7f4854104310 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f485b229700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485419ec70 con 0x7f48541019e0 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f485b229700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540ff7d0 con 0x7f4854106c40 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f485419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f485419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:45072/0 (socket says 172.21.15.64:45072) 2024-05-08T22:08:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 -- 172.21.15.64:0/748692274 learned_addr learned my addr 172.21.15.64:0/748692274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f4858fc5700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48541019e0 0x7f4854197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 -- 172.21.15.64:0/748692274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48541019e0 msgr2=0x7f4854197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48541019e0 0x7f4854197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 -- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4854104310 msgr2=0x7f4854197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4854104310 0x7f4854197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.626+0000 7f48597c6700 1 -- 172.21.15.64:0/748692274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c009d00 con 0x7f4854106c40 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.627+0000 7f48597c6700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f485419e560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f484400d980 tx=0x7f484400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.627+0000 7f4851ffb700 1 -- 172.21.15.64:0/748692274 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844004a30 con 0x7f4854106c40 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.627+0000 7f4851ffb700 1 -- 172.21.15.64:0/748692274 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484400b5c0 con 0x7f4854106c40 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.627+0000 7f4851ffb700 1 -- 172.21.15.64:0/748692274 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844009b50 con 0x7f4854106c40 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.627+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48540ffab0 con 0x7f4854106c40 2024-05-08T22:08:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.627+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48540fff70 con 0x7f4854106c40 2024-05-08T22:08:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.631+0000 7f4851ffb700 1 -- 172.21.15.64:0/748692274 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4844009cb0 con 0x7f4854106c40 2024-05-08T22:08:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.631+0000 7f4851ffb700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48400a1220 0x7f48400a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.631+0000 7f4851ffb700 1 -- 172.21.15.64:0/748692274 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48440cc740 con 0x7f4854106c40 2024-05-08T22:08:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.632+0000 7f4858fc5700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48400a1220 0x7f48400a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.632+0000 7f48673e3b80 1 Processor -- start 2024-05-08T22:08:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.633+0000 7f4858fc5700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48400a1220 0x7f48400a36e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f483c009c60 tx=0x7f483c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 -- start start 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d6290 0x5648105d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105d7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d7ae0 0x5648105d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105250f0 con 0x5648105d6bf0 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564810522ab0 con 0x5648105d7ae0 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f48673e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481051e3a0 con 0x5648105d6290 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f4853fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f4853fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105d7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49332/0 (socket says 172.21.15.64:49332) 2024-05-08T22:08:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.634+0000 7f4853fff700 1 -- 172.21.15.64:0/285382482 learned_addr learned my addr 172.21.15.64:0/285382482 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f4853fff700 1 -- 172.21.15.64:0/285382482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d6290 msgr2=0x5648105d66b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:08:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f4853fff700 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d6290 0x5648105d66b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f4853fff700 1 -- 172.21.15.64:0/285382482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d7ae0 msgr2=0x5648105d7f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:08:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f4853fff700 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d7ae0 0x5648105d7f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f4853fff700 1 -- 172.21.15.64:0/285382482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56481039f350 con 0x5648105d6bf0 2024-05-08T22:08:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f4853fff700 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105d7500 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b070 tx=0x7f484c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79246294a31fbed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f484affd700 1 -- 172.21.15.64:0/285382482 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00e040 con 0x5648105d6bf0 2024-05-08T22:08:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f484affd700 1 -- 172.21.15.64:0/285382482 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484c0076e0 con 0x5648105d6bf0 2024-05-08T22:08:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.635+0000 7f484affd700 1 -- 172.21.15.64:0/285382482 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c0052a0 con 0x5648105d6bf0 2024-05-08T22:08:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.637+0000 7f48673e3b80 1 -- 172.21.15.64:0/285382482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 msgr2=0x5648105d7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.637+0000 7f48673e3b80 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105d7500 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b070 tx=0x7f484c009be0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.641+0000 7f48673e3b80 1 -- 172.21.15.64:0/285382482 shutdown_connections 2024-05-08T22:08:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.641+0000 7f48673e3b80 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d7ae0 0x5648105d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.641+0000 7f48673e3b80 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d6290 0x5648105d66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.641+0000 7f48673e3b80 1 --2- 172.21.15.64:0/285382482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105d7500 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.641+0000 7f48673e3b80 1 -- 172.21.15.64:0/285382482 >> 172.21.15.64:0/285382482 conn(0x56481052c1b0 msgr2=0x5648105ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.641+0000 7f48673e3b80 1 -- 172.21.15.64:0/285382482 shutdown_connections 2024-05-08T22:08:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 -- 172.21.15.64:0/285382482 wait complete. 2024-05-08T22:08:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 -- start start 2024-05-08T22:08:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 Processor -- start 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648104ba500 con 0x5648105d6bf0 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648104ba1a0 con 0x5648105d6290 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.642+0000 7f48673e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648103a1640 con 0x5648105d7ae0 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f48673e3b80 10 client.0 _ll_register_callbacks cb 0x5648104c3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:56638/0 (socket says 172.21.15.64:56638) 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 -- 172.21.15.64:0/3087134635 learned_addr learned my addr 172.21.15.64:0/3087134635 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4853fff700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f48597c6700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 -- 172.21.15.64:0/3087134635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 msgr2=0x5648105f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 -- 172.21.15.64:0/3087134635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 msgr2=0x5648105f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 -- 172.21.15.64:0/3087134635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4854109c40 con 0x5648105d6290 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4853fff700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4858fc5700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105f1060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f483c00eb90 tx=0x7f483c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29587aba3a615579 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4833fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:08:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4833fff700 1 -- 172.21.15.64:0/3087134635 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c01cbc0 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.643+0000 7f4833fff700 1 -- 172.21.15.64:0/3087134635 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f483c023070 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564810477210 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f4833fff700 1 -- 172.21.15.64:0/3087134635 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f483c009e60 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 10 client.25949 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56481034bb00 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f4833fff700 1 -- 172.21.15.64:0/3087134635 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564810477210 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f4833fff700 1 -- 172.21.15.64:0/3087134635 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f483c031070 con 0x5648105d6290 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 20 client.25949 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648105cf2f0 0x5648105d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 4 client.25949 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.644+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56481059d7b0 con 0x5648105cf2f0 2024-05-08T22:08:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.645+0000 7f4853fff700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648105cf2f0 0x5648105d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.645+0000 7f4853fff700 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648105cf2f0 0x5648105d1710 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1b4df28ecb734e server_cookie=da339d28fc5ad929 in_seq=0 out_seq=0 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.645+0000 7f4833fff700 10 client.25949 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.646+0000 7f4833fff700 1 -- 172.21.15.64:0/3087134635 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56481059d7b0 con 0x5648105cf2f0 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.646+0000 7f4833fff700 10 client.25949 handle_command_reply: tid=0 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.646+0000 7f48673e3b80 1 client.25949 shutdown 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 msgr2=0x5648105f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105f1060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f483c00eb90 tx=0x7f483c0047c0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 shutdown_connections 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105f1060 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5648105cf2f0 0x5648105d1710 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 --2- 172.21.15.64:0/3087134635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 >> 172.21.15.64:0/3087134635 conn(0x56481052c1b0 msgr2=0x5648105c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f4833fff700 0 client.25949 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 shutdown_connections 2024-05-08T22:08:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 1 -- 172.21.15.64:0/3087134635 wait complete. 2024-05-08T22:08:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.647+0000 7f48673e3b80 20 client.25949 trim_cache size 0 max 16384 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 Processor -- start 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 -- start start 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105d7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105c37c0 con 0x5648105d6bf0 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105c3330 con 0x5648105d6290 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48673e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105efb50 con 0x5648105d7ae0 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f4853fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f4853fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49356/0 (socket says 172.21.15.64:49356) 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f4853fff700 1 -- 172.21.15.64:0/2586733107 learned_addr learned my addr 172.21.15.64:0/2586733107 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f4858fc5700 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105d7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.667+0000 7f48597c6700 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4853fff700 1 -- 172.21.15.64:0/2586733107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 msgr2=0x5648105d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4853fff700 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4853fff700 1 -- 172.21.15.64:0/2586733107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 msgr2=0x5648105d7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4853fff700 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105d7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4853fff700 1 -- 172.21.15.64:0/2586733107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648104e4500 con 0x5648105d6bf0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4858fc5700 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105d7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f4853fff700 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f85a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b0a0 tx=0x7f484c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9885854469ccfb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f484b7fe700 1 -- 172.21.15.64:0/2586733107 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00e040 con 0x5648105d6bf0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f484b7fe700 1 -- 172.21.15.64:0/2586733107 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484c0074a0 con 0x5648105d6bf0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.668+0000 7f484b7fe700 1 -- 172.21.15.64:0/2586733107 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c005900 con 0x5648105d6bf0 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 -- 172.21.15.64:0/2586733107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 msgr2=0x5648105f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f85a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b0a0 tx=0x7f484c0099e0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 -- 172.21.15.64:0/2586733107 shutdown_connections 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6290 0x5648105d7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 --2- 172.21.15.64:0/2586733107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6bf0 0x5648105f85a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 -- 172.21.15.64:0/2586733107 >> 172.21.15.64:0/2586733107 conn(0x56481052c1b0 msgr2=0x5648105f0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 -- 172.21.15.64:0/2586733107 shutdown_connections 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 -- 172.21.15.64:0/2586733107 wait complete. 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 -- start start 2024-05-08T22:08:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.669+0000 7f48673e3b80 1 Processor -- start 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.671+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6290 0x5648105f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.671+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6bf0 0x5648105f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.671+0000 7f48673e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.671+0000 7f48673e3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105f7b80 con 0x5648105d6290 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.671+0000 7f48673e3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105f7cf0 con 0x5648105d6bf0 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.671+0000 7f48673e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648105f7eb0 con 0x5648105d7ae0 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48673e3b80 10 client.0 _ll_register_callbacks cb 0x5648104c3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4853fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6bf0 0x5648105f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:45086/0 (socket says 172.21.15.64:45086) 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4853fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6bf0 0x5648105f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:56662/0 (socket says 172.21.15.64:56662) 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 -- 172.21.15.64:0/1031819506 learned_addr learned my addr 172.21.15.64:0/1031819506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4858fc5700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6290 0x5648105f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 -- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6bf0 msgr2=0x5648105f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6bf0 0x5648105f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 -- 172.21.15.64:0/1031819506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6290 msgr2=0x5648105f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6290 0x5648105f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 -- 172.21.15.64:0/1031819506 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56481034bb00 con 0x5648105d7ae0 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4858fc5700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6290 0x5648105f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48597c6700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f7640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48440965b0 tx=0x7f48440980f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f721a9d3a9a662ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4833fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4833fff700 1 -- 172.21.15.64:0/1031819506 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48440b9040 con 0x5648105d7ae0 2024-05-08T22:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f484c0076e0 con 0x5648105d7ae0 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.672+0000 7f4833fff700 1 -- 172.21.15.64:0/1031819506 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48440a1fc0 con 0x5648105d7ae0 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f4833fff700 1 -- 172.21.15.64:0/1031819506 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4844099230 con 0x5648105d7ae0 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 10 client.25798 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56481039f350 con 0x5648105d7ae0 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f4833fff700 1 -- 172.21.15.64:0/1031819506 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484c0076e0 con 0x5648105d7ae0 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f4833fff700 1 -- 172.21.15.64:0/1031819506 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f484409ab50 con 0x5648105d7ae0 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 10 client.25798 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 20 client.25798 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56481065bf40 0x56481065e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 4 client.25798 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564810516e60 con 0x56481065bf40 2024-05-08T22:08:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.673+0000 7f4858fc5700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56481065bf40 0x56481065e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.674+0000 7f4833fff700 10 client.25798 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.674+0000 7f4858fc5700 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56481065bf40 0x56481065e360 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1972c68516c3cd33 server_cookie=f59fdf6b4f8e65a9 in_seq=0 out_seq=0 2024-05-08T22:08:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f4833fff700 1 -- 172.21.15.64:0/1031819506 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564810516e60 con 0x56481065bf40 2024-05-08T22:08:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f4833fff700 10 client.25798 handle_command_reply: tid=0 2024-05-08T22:08:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 client.25798 shutdown 2024-05-08T22:08:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 msgr2=0x5648105f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f7640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48440965b0 tx=0x7f48440980f0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 shutdown_connections 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56481065bf40 0x56481065e360 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648105d7ae0 0x5648105f7640 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648105d6290 0x5648105f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 --2- 172.21.15.64:0/1031819506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648105d6bf0 0x5648105f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f4833fff700 0 client.25798 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 >> 172.21.15.64:0/1031819506 conn(0x56481052c1b0 msgr2=0x56481060db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 shutdown_connections 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.683+0000 7f48673e3b80 1 -- 172.21.15.64:0/1031819506 wait complete. 2024-05-08T22:08:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f48673e3b80 20 client.25798 trim_cache size 0 max 16384 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48400a1220 msgr2=0x7f48400a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48400a1220 0x7f48400a36e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f483c009c60 tx=0x7f483c009260 comp rx=0 tx=0).stop 2024-05-08T22:08:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 msgr2=0x7f485419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f485419e560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f484400d980 tx=0x7f484400dc90 comp rx=0 tx=0).stop 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 shutdown_connections 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f48400a1220 0x7f48400a36e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48541019e0 0x7f4854197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4854106c40 0x7f485419e560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 --2- 172.21.15.64:0/748692274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4854104310 0x7f4854197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 >> 172.21.15.64:0/748692274 conn(0x7f48540fb390 msgr2=0x7f4854103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 shutdown_connections 2024-05-08T22:08:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:20.684+0000 7f485b229700 1 -- 172.21.15.64:0/748692274 wait complete. 2024-05-08T22:08:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 810.0 s, 1602.3 tps, lat 62.936 ms stddev 73.342 2024-05-08T22:08:21.964 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:21 smithi064 ceph-mon[42500]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.51k op/s 2024-05-08T22:08:22.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:21 smithi160 ceph-mon[46962]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.51k op/s 2024-05-08T22:08:22.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:21 smithi186 ceph-mon[46619]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.51k op/s 2024-05-08T22:08:23.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:23 smithi064 ceph-mon[42500]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 973 op/s 2024-05-08T22:08:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:23 smithi160 ceph-mon[46962]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 973 op/s 2024-05-08T22:08:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:23 smithi186 ceph-mon[46619]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 973 op/s 2024-05-08T22:08:25.969 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:25 smithi064 ceph-mon[42500]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-08T22:08:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:25 smithi160 ceph-mon[46962]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-08T22:08:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:25 smithi186 ceph-mon[46619]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-08T22:08:26.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 815.0 s, 1448.4 tps, lat 68.663 ms stddev 72.348 2024-05-08T22:08:27.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:27 smithi064 ceph-mon[42500]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 647 op/s 2024-05-08T22:08:28.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:27 smithi160 ceph-mon[46962]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 647 op/s 2024-05-08T22:08:28.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:27 smithi186 ceph-mon[46619]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 647 op/s 2024-05-08T22:08:29.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:29 smithi064 ceph-mon[42500]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 615 op/s 2024-05-08T22:08:30.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:29 smithi160 ceph-mon[46962]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 615 op/s 2024-05-08T22:08:30.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:29 smithi186 ceph-mon[46619]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 615 op/s 2024-05-08T22:08:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 820.0 s, 1591.0 tps, lat 62.017 ms stddev 66.838 2024-05-08T22:08:31.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:31 smithi064 ceph-mon[42500]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 641 op/s 2024-05-08T22:08:32.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:31 smithi160 ceph-mon[46962]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 641 op/s 2024-05-08T22:08:32.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:31 smithi186 ceph-mon[46619]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 641 op/s 2024-05-08T22:08:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/2619158108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48101410 msgr2=0x7f7d48101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/2619158108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48101410 0x7f7d48101830 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009990 tx=0x7f7d38009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/2619158108 shutdown_connections 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/2619158108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48100a20 0x7f7d481082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/2619158108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48101410 0x7f7d48101830 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/2619158108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 0x7f7d481004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/2619158108 >> 172.21.15.64:0/2619158108 conn(0x7f7d480faaf0 msgr2=0x7f7d480fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/2619158108 shutdown_connections 2024-05-08T22:08:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.841+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/2619158108 wait complete. 2024-05-08T22:08:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.842+0000 7f7d4c8e5700 1 Processor -- start 2024-05-08T22:08:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 -- start start 2024-05-08T22:08:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 0x7f7d480ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100a20 0x7f7d480714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 0x7f7d48071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a60f0 con 0x7f7d48100a20 2024-05-08T22:08:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a6260 con 0x7f7d48100060 2024-05-08T22:08:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4c8e5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a6420 con 0x7f7d48101410 2024-05-08T22:08:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 0x7f7d48071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d4659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 0x7f7d480ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 0x7f7d48071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:51840/0 (socket says 172.21.15.64:51840) 2024-05-08T22:08:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.843+0000 7f7d46d9d700 1 -- 172.21.15.64:0/1687570778 learned_addr learned my addr 172.21.15.64:0/1687570778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100a20 0x7f7d480714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4659c700 1 -- 172.21.15.64:0/1687570778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 msgr2=0x7f7d48071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4659c700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 0x7f7d48071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4659c700 1 -- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100a20 msgr2=0x7f7d480714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4659c700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100a20 0x7f7d480714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4659c700 1 -- 172.21.15.64:0/1687570778 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380095f0 con 0x7f7d48100060 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 0x7f7d48071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4659c700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 0x7f7d480ffad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d38004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/1687570778 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3801d070 con 0x7f7d48100060 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/1687570778 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d38021410 con 0x7f7d48100060 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d48072020 con 0x7f7d48100060 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d480724d0 con 0x7f7d48100060 2024-05-08T22:08:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.844+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/1687570778 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38021ea0 con 0x7f7d48100060 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 Processor -- start 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 -- start start 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/1687570778 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7d3800fb10 con 0x7f7d48100060 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fb7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb83e0 0x561827fb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb92d0 0x561827fb9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827f0f0f0 con 0x561827fb7a80 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827f0cab0 con 0x561827fb92d0 2024-05-08T22:08:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827f083a0 con 0x561827fb83e0 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fb7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fb7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54896/0 (socket says 172.21.15.64:54896) 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3479912147 learned_addr learned my addr 172.21.15.64:0/3479912147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d4659c700 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb83e0 0x561827fb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.849+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb92d0 0x561827fb9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3479912147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb83e0 msgr2=0x561827fb8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb83e0 0x561827fb8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3479912147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb92d0 msgr2=0x561827fb9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb92d0 0x561827fb9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3479912147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561827d89350 con 0x561827fb7a80 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d3f7fe700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d340a1120 0x7f7d340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fb7ea0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009500 tx=0x7f7d40009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e437cc3785a52ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d3cff9700 1 -- 172.21.15.64:0/3479912147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000e900 con 0x561827fb7a80 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/1687570778 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d380cd8a0 con 0x7f7d48100060 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d3cff9700 1 -- 172.21.15.64:0/3479912147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d4000f040 con 0x561827fb7a80 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d3cff9700 1 -- 172.21.15.64:0/3479912147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40017d20 con 0x561827fb7a80 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.850+0000 7f7d4659c700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d340a1120 0x7f7d340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.851+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3479912147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 msgr2=0x561827fb7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.851+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fb7ea0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009500 tx=0x7f7d40009920 comp rx=0 tx=0).stop 2024-05-08T22:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.851+0000 7f7d4659c700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d340a1120 0x7f7d340a35e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7d3809aa90 tx=0x7f7d380d3550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.852+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3479912147 shutdown_connections 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.852+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb92d0 0x561827fb9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.852+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fb7ea0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.852+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3479912147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb83e0 0x561827fb8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.852+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3479912147 >> 172.21.15.64:0/3479912147 conn(0x561827f161b0 msgr2=0x561827fb4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.853+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3479912147 shutdown_connections 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.853+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3479912147 wait complete. 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.853+0000 7f7d58a9fb80 1 -- start start 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.853+0000 7f7d58a9fb80 1 Processor -- start 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561827fc8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827ea4500 con 0x561827fb7a80 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827ea41a0 con 0x561827fb83e0 2024-05-08T22:08:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827d8b640 con 0x561827fb92d0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 10 client.0 _ll_register_callbacks cb 0x561827eadad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54898/0 (socket says 172.21.15.64:54898) 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 -- 172.21.15.64:0/4245749047 learned_addr learned my addr 172.21.15.64:0/4245749047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561827fc8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 -- 172.21.15.64:0/4245749047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 msgr2=0x561827fc8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561827fc8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 -- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 msgr2=0x561827fc1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 -- 172.21.15.64:0/4245749047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d400091e0 con 0x561827fb7a80 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc18c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7d40007ba0 tx=0x7f7d40009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f007ba5b569c794c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:08:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d2dffb700 1 -- 172.21.15.64:0/4245749047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000e910 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d2dffb700 1 -- 172.21.15.64:0/4245749047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d4000ea70 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.854+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561827e61210 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d2dffb700 1 -- 172.21.15.64:0/4245749047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d4000ec30 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 10 client.16173 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561827d35b00 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d2dffb700 1 -- 172.21.15.64:0/4245749047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561827e61210 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d2dffb700 1 -- 172.21.15.64:0/4245749047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7d4002ebc0 con 0x561827fb7a80 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 20 client.16173 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561827fcd6d0 0x561827fcfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 4 client.16173 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.855+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561827f877b0 con 0x561827fcd6d0 2024-05-08T22:08:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.857+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561827fcd6d0 0x561827fcfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.857+0000 7f7d2dffb700 10 client.16173 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.857+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561827fcd6d0 0x561827fcfaf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22dac08003f29fb server_cookie=f203b0985872dff7 in_seq=0 out_seq=0 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d2dffb700 1 -- 172.21.15.64:0/4245749047 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561827f877b0 con 0x561827fcd6d0 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d2dffb700 10 client.16173 handle_command_reply: tid=0 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 client.16173 shutdown 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 msgr2=0x561827fc18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc18c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7d40007ba0 tx=0x7f7d40009e90 comp rx=0 tx=0).stop 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 shutdown_connections 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561827fcd6d0 0x561827fcfaf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561827fc8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc18c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/4245749047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 >> 172.21.15.64:0/4245749047 conn(0x561827f161b0 msgr2=0x561827fb2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d2dffb700 0 client.16173 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 shutdown_connections 2024-05-08T22:08:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/4245749047 wait complete. 2024-05-08T22:08:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.858+0000 7f7d58a9fb80 20 client.16173 trim_cache size 0 max 16384 2024-05-08T22:08:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.875+0000 7f7d58a9fb80 1 Processor -- start 2024-05-08T22:08:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 -- start start 2024-05-08T22:08:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb7a80 0x561827fb96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb92d0 0x561827fc0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827fc9fc0 con 0x561827fb92d0 2024-05-08T22:08:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827fc0390 con 0x561827fb83e0 2024-05-08T22:08:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.876+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827fc2ab0 con 0x561827fb7a80 2024-05-08T22:08:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.877+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.878+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb7a80 0x561827fb96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.878+0000 7f7d46d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb7a80 0x561827fb96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:51860/0 (socket says 172.21.15.64:51860) 2024-05-08T22:08:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.878+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3377664708 learned_addr learned my addr 172.21.15.64:0/3377664708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.879+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb92d0 0x561827fc0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.879+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3377664708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb7a80 msgr2=0x561827fb96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.879+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb7a80 0x561827fb96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.879+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3377664708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 msgr2=0x561827fc8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.879+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.879+0000 7f7d46d9d700 1 -- 172.21.15.64:0/3377664708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561827ece500 con 0x561827fb92d0 2024-05-08T22:08:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.880+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb92d0 0x561827fc0dd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7d40017830 tx=0x7f7d40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef00707c4fb81b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.882+0000 7f7d3d7fa700 1 -- 172.21.15.64:0/3377664708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40020610 con 0x561827fb92d0 2024-05-08T22:08:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.882+0000 7f7d3d7fa700 1 -- 172.21.15.64:0/3377664708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d40017d90 con 0x561827fb92d0 2024-05-08T22:08:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.882+0000 7f7d3d7fa700 1 -- 172.21.15.64:0/3377664708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40020ab0 con 0x561827fb92d0 2024-05-08T22:08:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.883+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3377664708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb92d0 msgr2=0x561827fc0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.883+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb92d0 0x561827fc0dd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7d40017830 tx=0x7f7d40009bd0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.895+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3377664708 shutdown_connections 2024-05-08T22:08:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.895+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb92d0 0x561827fc0dd0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.895+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb7a80 0x561827fb96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.895+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/3377664708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.895+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3377664708 >> 172.21.15.64:0/3377664708 conn(0x561827f161b0 msgr2=0x561827fbf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.900+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3377664708 shutdown_connections 2024-05-08T22:08:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.911+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/3377664708 wait complete. 2024-05-08T22:08:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.911+0000 7f7d58a9fb80 1 -- start start 2024-05-08T22:08:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.911+0000 7f7d58a9fb80 1 Processor -- start 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561828021b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561827fc6c90 con 0x561827fb7a80 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618280220e0 con 0x561827fb83e0 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618280222a0 con 0x561827fb92d0 2024-05-08T22:08:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.915+0000 7f7d58a9fb80 10 client.0 _ll_register_callbacks cb 0x561827eadad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.920+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.920+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:50620/0 (socket says 172.21.15.64:50620) 2024-05-08T22:08:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.920+0000 7f7d3ffff700 1 -- 172.21.15.64:0/546904555 learned_addr learned my addr 172.21.15.64:0/546904555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.923+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561828021b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.923+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.938+0000 7f7d46d9d700 1 -- 172.21.15.64:0/546904555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 msgr2=0x561828021b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.938+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561828021b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.938+0000 7f7d46d9d700 1 -- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 msgr2=0x561827fc6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.938+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.938+0000 7f7d46d9d700 1 -- 172.21.15.64:0/546904555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561827d35b00 con 0x561827fb7a80 2024-05-08T22:08:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.939+0000 7f7d46d9d700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc5fe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7d400061e0 tx=0x7f7d40009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c991db39e9f5a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.940+0000 7f7d2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.947+0000 7f7d2dffb700 1 -- 172.21.15.64:0/546904555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40020a60 con 0x561827fb7a80 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.947+0000 7f7d2dffb700 1 -- 172.21.15.64:0/546904555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d4003f060 con 0x561827fb7a80 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.947+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d4000f040 con 0x561827fb7a80 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d2dffb700 1 -- 172.21.15.64:0/546904555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d40020bc0 con 0x561827fb7a80 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d58a9fb80 10 client.16179 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561827d8dc70 con 0x561827fb7a80 2024-05-08T22:08:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d2dffb700 1 -- 172.21.15.64:0/546904555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d4000f040 con 0x561827fb7a80 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d2dffb700 1 -- 172.21.15.64:0/546904555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7d40005830 con 0x561827fb7a80 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d58a9fb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d58a9fb80 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d58a9fb80 20 client.16179 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.948+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561828023790 0x561828025b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.949+0000 7f7d58a9fb80 4 client.16179 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.949+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561827f00e60 con 0x561828023790 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.950+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561828023790 0x561828025b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.950+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561828023790 0x561828025b80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3703dfd2e2a867b9 server_cookie=579aea7918c5126f in_seq=0 out_seq=0 2024-05-08T22:08:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.950+0000 7f7d2dffb700 10 client.16179 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d2dffb700 1 -- 172.21.15.64:0/546904555 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561827f00e60 con 0x561828023790 2024-05-08T22:08:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d2dffb700 10 client.16179 handle_command_reply: tid=0 2024-05-08T22:08:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 client.16179 shutdown 2024-05-08T22:08:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 msgr2=0x561827fc5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc5fe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7d400061e0 tx=0x7f7d40009a50 comp rx=0 tx=0).stop 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 shutdown_connections 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x561828023790 0x561828025b80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561827fb92d0 0x561828021b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561827fb7a80 0x561827fc5fe0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 --2- 172.21.15.64:0/546904555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561827fb83e0 0x561827fc6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 >> 172.21.15.64:0/546904555 conn(0x561827f161b0 msgr2=0x561827fbc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d2dffb700 0 client.16179 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 shutdown_connections 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 1 -- 172.21.15.64:0/546904555 wait complete. 2024-05-08T22:08:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.976+0000 7f7d58a9fb80 20 client.16179 trim_cache size 0 max 16384 2024-05-08T22:08:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:08:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:08:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:08:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d340a1120 msgr2=0x7f7d340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d340a1120 0x7f7d340a35e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7d3809aa90 tx=0x7f7d380d3550 comp rx=0 tx=0).stop 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 msgr2=0x7f7d480ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 0x7f7d480ffad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d38004730 comp rx=0 tx=0).stop 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 shutdown_connections 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7d340a1120 0x7f7d340a35e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48100060 0x7f7d480ffad0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d48101410 0x7f7d48071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 --2- 172.21.15.64:0/1687570778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d48100a20 0x7f7d480714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 >> 172.21.15.64:0/1687570778 conn(0x7f7d480faaf0 msgr2=0x7f7d481044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 shutdown_connections 2024-05-08T22:08:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:33.977+0000 7f7d4c8e5700 1 -- 172.21.15.64:0/1687570778 wait complete. 2024-05-08T22:08:34.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:33 smithi160 ceph-mon[46962]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-05-08T22:08:34.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:33 smithi186 ceph-mon[46619]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-05-08T22:08:34.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:33 smithi064 ceph-mon[42500]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-05-08T22:08:35.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:34 smithi186 ceph-mon[46619]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-05-08T22:08:35.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:34 smithi064 ceph-mon[42500]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-05-08T22:08:35.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:34 smithi160 ceph-mon[46962]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-05-08T22:08:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 825.0 s, 1797.2 tps, lat 56.384 ms stddev 65.088 2024-05-08T22:08:37.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:37 smithi064 ceph-mon[42500]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 677 op/s 2024-05-08T22:08:38.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:37 smithi160 ceph-mon[46962]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 677 op/s 2024-05-08T22:08:38.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:37 smithi186 ceph-mon[46619]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 677 op/s 2024-05-08T22:08:39.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:39 smithi064 ceph-mon[42500]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 MiB/s wr, 654 op/s 2024-05-08T22:08:40.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:39 smithi160 ceph-mon[46962]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 MiB/s wr, 654 op/s 2024-05-08T22:08:40.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:39 smithi186 ceph-mon[46619]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 MiB/s wr, 654 op/s 2024-05-08T22:08:40.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:40 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:08:41.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:40 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:08:41.150 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:40 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:08:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 830.0 s, 1359.8 tps, lat 73.408 ms stddev 87.577 2024-05-08T22:08:41.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:41 smithi064 ceph-mon[42500]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 MiB/s wr, 687 op/s 2024-05-08T22:08:41.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:41.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:41 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:42.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:41 smithi160 ceph-mon[46962]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 MiB/s wr, 687 op/s 2024-05-08T22:08:42.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:42.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:41 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:41 smithi186 ceph-mon[46619]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 MiB/s wr, 687 op/s 2024-05-08T22:08:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:42.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:41 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:42.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:08:42.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:08:42.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:43.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:08:43.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:08:43.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:08:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:08:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:08:43.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:43 smithi064 ceph-mon[42500]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 MiB/s wr, 705 op/s 2024-05-08T22:08:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:43 smithi160 ceph-mon[46962]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 MiB/s wr, 705 op/s 2024-05-08T22:08:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:43 smithi186 ceph-mon[46619]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 MiB/s wr, 705 op/s 2024-05-08T22:08:46.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:45 smithi160 ceph-mon[46962]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.9 MiB/s wr, 683 op/s 2024-05-08T22:08:46.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:45 smithi186 ceph-mon[46619]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.9 MiB/s wr, 683 op/s 2024-05-08T22:08:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:45 smithi064 ceph-mon[42500]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.9 MiB/s wr, 683 op/s 2024-05-08T22:08:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 835.0 s, 1621.6 tps, lat 61.286 ms stddev 70.372 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.130+0000 7f23cf52a700 1 -- 172.21.15.64:0/746962002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c80ffe40 msgr2=0x7f23c8100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.130+0000 7f23cf52a700 1 --2- 172.21.15.64:0/746962002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c80ffe40 0x7f23c8100260 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009990 tx=0x7f23c4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 -- 172.21.15.64:0/746962002 shutdown_connections 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 --2- 172.21.15.64:0/746962002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c8101950 0x7f23c8108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 --2- 172.21.15.64:0/746962002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c80ffe40 0x7f23c8100260 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 --2- 172.21.15.64:0/746962002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c8100f90 0x7f23c8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 -- 172.21.15.64:0/746962002 >> 172.21.15.64:0/746962002 conn(0x7f23c80fb3b0 msgr2=0x7f23c80fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 -- 172.21.15.64:0/746962002 shutdown_connections 2024-05-08T22:08:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 -- 172.21.15.64:0/746962002 wait complete. 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 Processor -- start 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.131+0000 7f23cf52a700 1 -- start start 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cf52a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 0x7f23c8196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cf52a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c8100f90 0x7f23c81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cf52a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c8101950 0x7f23c819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cf52a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c819e9c0 con 0x7f23c8101950 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cf52a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c819eb00 con 0x7f23c8100f90 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cf52a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c819ec90 con 0x7f23c80ffe40 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cdac7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c8101950 0x7f23c819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 0x7f23c8196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 0x7f23c8196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:57350/0 (socket says 172.21.15.64:57350) 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1454805100 learned_addr learned my addr 172.21.15.64:0/1454805100 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cdac7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c8101950 0x7f23c819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47370/0 (socket says 172.21.15.64:47370) 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23ccac5700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c8100f90 0x7f23c81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1454805100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c8100f90 msgr2=0x7f23c81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c8100f90 0x7f23c81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c8101950 msgr2=0x7f23c819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c8101950 0x7f23c819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1454805100 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40095f0 con 0x7f23c80ffe40 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.132+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 0x7f23c8196e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23c4004760 tx=0x7f23c4004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.133+0000 7f23be7fc700 1 -- 172.21.15.64:0/1454805100 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c401d070 con 0x7f23c80ffe40 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.133+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c8102240 con 0x7f23c80ffe40 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.133+0000 7f23be7fc700 1 -- 172.21.15.64:0/1454805100 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c4004830 con 0x7f23c80ffe40 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.133+0000 7f23be7fc700 1 -- 172.21.15.64:0/1454805100 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4005420 con 0x7f23c80ffe40 2024-05-08T22:08:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.133+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c81026e0 con 0x7f23c80ffe40 2024-05-08T22:08:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.135+0000 7f23be7fc700 1 -- 172.21.15.64:0/1454805100 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f23c4005580 con 0x7f23c80ffe40 2024-05-08T22:08:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.136+0000 7f23be7fc700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23b40a1120 0x7f23b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.136+0000 7f23ccac5700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23b40a1120 0x7f23b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.136+0000 7f23be7fc700 1 -- 172.21.15.64:0/1454805100 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23c40cd950 con 0x7f23c80ffe40 2024-05-08T22:08:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.137+0000 7f23ccac5700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23b40a1120 0x7f23b40a35e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009c60 tx=0x7f23b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 Processor -- start 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 -- start start 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95234ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95235cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952362d0 0x560a95236750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9518c0f0 con 0x560a952362d0 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95189ab0 con 0x560a95234a80 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a951853a0 con 0x560a952353e0 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23cdac7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952362d0 0x560a95236750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23cdac7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952362d0 0x560a95236750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47384/0 (socket says 172.21.15.64:47384) 2024-05-08T22:08:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.141+0000 7f23cdac7700 1 -- 172.21.15.64:0/1724026701 learned_addr learned my addr 172.21.15.64:0/1724026701 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95234ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23ccac5700 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95235cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1724026701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 msgr2=0x560a95235cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95235cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1724026701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952362d0 msgr2=0x560a95236750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952362d0 0x560a95236750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 -- 172.21.15.64:0/1724026701 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a95006350 con 0x560a95234a80 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95234ea0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23c40cb4b0 tx=0x7f23c409a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9a20adaf46e9a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23b37fe700 1 -- 172.21.15.64:0/1724026701 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c409ec70 con 0x560a95234a80 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23b37fe700 1 -- 172.21.15.64:0/1724026701 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c40a3230 con 0x560a95234a80 2024-05-08T22:08:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.142+0000 7f23b37fe700 1 -- 172.21.15.64:0/1724026701 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a3000 con 0x560a95234a80 2024-05-08T22:08:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 -- 172.21.15.64:0/1724026701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 msgr2=0x560a95234ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95234ea0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23c40cb4b0 tx=0x7f23c409a870 comp rx=0 tx=0).stop 2024-05-08T22:08:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 -- 172.21.15.64:0/1724026701 shutdown_connections 2024-05-08T22:08:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95234ea0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952362d0 0x560a95236750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/1724026701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95235cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 -- 172.21.15.64:0/1724026701 >> 172.21.15.64:0/1724026701 conn(0x560a951931b0 msgr2=0x560a95231440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 -- 172.21.15.64:0/1724026701 shutdown_connections 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 -- 172.21.15.64:0/1724026701 wait complete. 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 -- start start 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.143+0000 7f23db6e4b80 1 Processor -- start 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a9523e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952353e0 0x560a9523ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a95245ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95121500 con 0x560a95234a80 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a951211a0 con 0x560a952353e0 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95008640 con 0x560a952362d0 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23db6e4b80 10 client.0 _ll_register_callbacks cb 0x560a9512aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23cd2c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a9523e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23cd2c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a9523e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47388/0 (socket says 172.21.15.64:47388) 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23cd2c6700 1 -- 172.21.15.64:0/2799182020 learned_addr learned my addr 172.21.15.64:0/2799182020 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23ccac5700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952353e0 0x560a9523ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23cdac7700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a95245ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23ccac5700 1 -- 172.21.15.64:0/2799182020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 msgr2=0x560a95245ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23ccac5700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a95245ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23ccac5700 1 -- 172.21.15.64:0/2799182020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 msgr2=0x560a9523e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23ccac5700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a9523e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.144+0000 7f23ccac5700 1 -- 172.21.15.64:0/2799182020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c81026e0 con 0x560a952353e0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23ccac5700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952353e0 0x560a9523ea80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f23b8011ab0 tx=0x7f23b80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e51c8f37d7a07d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23b0ff9700 1 -- 172.21.15.64:0/2799182020 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8022070 con 0x560a952353e0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23b0ff9700 1 -- 172.21.15.64:0/2799182020 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23b801d620 con 0x560a952353e0 2024-05-08T22:08:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a950de210 con 0x560a952353e0 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23b0ff9700 1 -- 172.21.15.64:0/2799182020 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b801d7e0 con 0x560a952353e0 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23db6e4b80 10 client.25982 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560a94fb2b00 con 0x560a952353e0 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23b0ff9700 1 -- 172.21.15.64:0/2799182020 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a950de210 con 0x560a952353e0 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23b0ff9700 1 -- 172.21.15.64:0/2799182020 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23b8027b00 con 0x560a952353e0 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23db6e4b80 10 client.25982 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.145+0000 7f23db6e4b80 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23db6e4b80 20 client.25982 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a9524a380 0x560a9524c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23db6e4b80 4 client.25982 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a952047b0 con 0x560a9524a380 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a9524a380 0x560a9524c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23b0ff9700 10 client.25982 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.146+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a9524a380 0x560a9524c7a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba92083aa876098 server_cookie=57fa4be503318009 in_seq=0 out_seq=0 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23b0ff9700 1 -- 172.21.15.64:0/2799182020 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a952047b0 con 0x560a9524a380 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23b0ff9700 10 client.25982 handle_command_reply: tid=0 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 client.25982 shutdown 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952353e0 msgr2=0x560a9523ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952353e0 0x560a9523ea80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f23b8011ab0 tx=0x7f23b80047c0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 shutdown_connections 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a9524a380 0x560a9524c7a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a95245ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a9523e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/2799182020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952353e0 0x560a9523ea80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 >> 172.21.15.64:0/2799182020 conn(0x560a951931b0 msgr2=0x560a9522f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.147+0000 7f23b0ff9700 0 client.25982 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.148+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 shutdown_connections 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.148+0000 7f23db6e4b80 1 -- 172.21.15.64:0/2799182020 wait complete. 2024-05-08T22:08:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.148+0000 7f23db6e4b80 20 client.25982 trim_cache size 0 max 16384 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 Processor -- start 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 -- start start 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a95246170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95245a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952362d0 0x560a952366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95246c40 con 0x560a95234a80 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9523e0b0 con 0x560a952362d0 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95230960 con 0x560a952353e0 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23cd2c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a95246170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23ccac5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95245a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23cd2c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a95246170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47392/0 (socket says 172.21.15.64:47392) 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23cd2c6700 1 -- 172.21.15.64:0/3744526312 learned_addr learned my addr 172.21.15.64:0/3744526312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23ccac5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95245a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:57368/0 (socket says 172.21.15.64:57368) 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.165+0000 7f23cdac7700 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952362d0 0x560a952366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cd2c6700 1 -- 172.21.15.64:0/3744526312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 msgr2=0x560a95245a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95245a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cd2c6700 1 -- 172.21.15.64:0/3744526312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952362d0 msgr2=0x560a952366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952362d0 0x560a952366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cd2c6700 1 -- 172.21.15.64:0/3744526312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a9514b500 con 0x560a95234a80 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23ccac5700 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95245a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cdac7700 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952362d0 0x560a952366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a95246170 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f23c4097340 tx=0x7f23c40cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7c922b9ad03a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23b3fff700 1 -- 172.21.15.64:0/3744526312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c409ec70 con 0x560a95234a80 2024-05-08T22:08:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23b3fff700 1 -- 172.21.15.64:0/3744526312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c40991e0 con 0x560a95234a80 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.166+0000 7f23b3fff700 1 -- 172.21.15.64:0/3744526312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a3000 con 0x560a95234a80 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 -- 172.21.15.64:0/3744526312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 msgr2=0x560a95246170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a95246170 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f23c4097340 tx=0x7f23c40cd040 comp rx=0 tx=0).stop 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 -- 172.21.15.64:0/3744526312 shutdown_connections 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a952362d0 0x560a952366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a95234a80 0x560a95246170 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/3744526312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952353e0 0x560a95245a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 -- 172.21.15.64:0/3744526312 >> 172.21.15.64:0/3744526312 conn(0x560a951931b0 msgr2=0x560a95239510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 -- 172.21.15.64:0/3744526312 shutdown_connections 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.167+0000 7f23db6e4b80 1 -- 172.21.15.64:0/3744526312 wait complete. 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 -- start start 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 Processor -- start 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95242c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 0x560a952431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a9529e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95243720 con 0x560a952353e0 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a95243890 con 0x560a95234a80 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9529edd0 con 0x560a952362d0 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23db6e4b80 10 client.0 _ll_register_callbacks cb 0x560a9512aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23ccac5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 0x560a952431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23ccac5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 0x560a952431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47402/0 (socket says 172.21.15.64:47402) 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23ccac5700 1 -- 172.21.15.64:0/241505334 learned_addr learned my addr 172.21.15.64:0/241505334 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23cdac7700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a9529e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.168+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95242c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23ccac5700 1 -- 172.21.15.64:0/241505334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 msgr2=0x560a9529e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23ccac5700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a9529e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23ccac5700 1 -- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 msgr2=0x560a95242c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23ccac5700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95242c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23ccac5700 1 -- 172.21.15.64:0/241505334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a94fb2b00 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23cdac7700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a9529e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23ccac5700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 0x560a952431a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009b20 tx=0x7f23b8031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3288e9f32b4bb060 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23b0ff9700 1 -- 172.21.15.64:0/241505334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8037070 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23b0ff9700 1 -- 172.21.15.64:0/241505334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23b80045c0 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23c40a3230 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23b0ff9700 1 -- 172.21.15.64:0/241505334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b8022040 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23db6e4b80 10 client.16197 fetch_fsmap learned FSMap version 47 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f23c0003ca0 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.169+0000 7f23b0ff9700 1 -- 172.21.15.64:0/241505334 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c40a3230 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23b0ff9700 1 -- 172.21.15.64:0/241505334 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23b8045070 con 0x560a952353e0 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23db6e4b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:08:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23db6e4b80 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23db6e4b80 20 client.16197 populate_metadata read hostname 'smithi064' 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a952a07e0 0x560a952a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23db6e4b80 4 client.16197 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a9517de60 con 0x560a952a07e0 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a952a07e0 0x560a952a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23b0ff9700 10 client.16197 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.170+0000 7f23cd2c6700 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a952a07e0 0x560a952a2c00 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4588792e6f7720dd server_cookie=6b72d0904febb015 in_seq=0 out_seq=0 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23b0ff9700 1 -- 172.21.15.64:0/241505334 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a9517de60 con 0x560a952a07e0 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23b0ff9700 10 client.16197 handle_command_reply: tid=0 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 client.16197 shutdown 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 msgr2=0x560a952431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 0x560a952431a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009b20 tx=0x7f23b8031cd0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 shutdown_connections 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a95234a80 0x560a95242c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x560a952a07e0 0x560a952a2c00 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a952362d0 0x560a9529e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 --2- 172.21.15.64:0/241505334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a952353e0 0x560a952431a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23b0ff9700 0 client.16197 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 >> 172.21.15.64:0/241505334 conn(0x560a951931b0 msgr2=0x560a9524c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 shutdown_connections 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.178+0000 7f23db6e4b80 1 -- 172.21.15.64:0/241505334 wait complete. 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23db6e4b80 20 client.16197 trim_cache size 0 max 16384 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:08:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:08:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23b40a1120 msgr2=0x7f23b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23b40a1120 0x7f23b40a35e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009c60 tx=0x7f23b8009260 comp rx=0 tx=0).stop 2024-05-08T22:08:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 msgr2=0x7f23c8196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 0x7f23c8196e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23c4004760 tx=0x7f23c4004790 comp rx=0 tx=0).stop 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 shutdown_connections 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23b40a1120 0x7f23b40a35e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c8101950 0x7f23c819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c80ffe40 0x7f23c8196e90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 --2- 172.21.15.64:0/1454805100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c8100f90 0x7f23c81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 >> 172.21.15.64:0/1454805100 conn(0x7f23c80fb3b0 msgr2=0x7f23c8069160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 shutdown_connections 2024-05-08T22:08:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:08:47.179+0000 7f23cf52a700 1 -- 172.21.15.64:0/1454805100 wait complete. 2024-05-08T22:08:48.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:47 smithi160 ceph-mon[46962]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.4 MiB/s wr, 684 op/s 2024-05-08T22:08:48.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:47 smithi186 ceph-mon[46619]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.4 MiB/s wr, 684 op/s 2024-05-08T22:08:48.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:47 smithi064 ceph-mon[42500]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.4 MiB/s wr, 684 op/s 2024-05-08T22:08:50.056 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:49 smithi160 ceph-mon[46962]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 MiB/s wr, 674 op/s 2024-05-08T22:08:50.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:49 smithi186 ceph-mon[46619]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 MiB/s wr, 674 op/s 2024-05-08T22:08:50.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:49 smithi064 ceph-mon[42500]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 MiB/s wr, 674 op/s 2024-05-08T22:08:51.483 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 840.0 s, 1892.2 tps, lat 53.011 ms stddev 65.087 2024-05-08T22:08:52.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:51 smithi160 ceph-mon[46962]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 727 op/s 2024-05-08T22:08:52.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:51 smithi186 ceph-mon[46619]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 727 op/s 2024-05-08T22:08:52.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:51 smithi064 ceph-mon[42500]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 727 op/s 2024-05-08T22:08:54.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:53 smithi160 ceph-mon[46962]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 MiB/s wr, 725 op/s 2024-05-08T22:08:54.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:53 smithi186 ceph-mon[46619]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 MiB/s wr, 725 op/s 2024-05-08T22:08:54.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:53 smithi064 ceph-mon[42500]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 MiB/s wr, 725 op/s 2024-05-08T22:08:56.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:55 smithi160 ceph-mon[46962]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 748 op/s 2024-05-08T22:08:56.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:55 smithi186 ceph-mon[46619]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 748 op/s 2024-05-08T22:08:56.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:55 smithi064 ceph-mon[42500]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 748 op/s 2024-05-08T22:08:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 845.0 s, 1107.8 tps, lat 90.764 ms stddev 110.915 2024-05-08T22:08:57.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:56 smithi160 ceph-mon[46962]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 763 op/s 2024-05-08T22:08:57.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:56 smithi186 ceph-mon[46619]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 763 op/s 2024-05-08T22:08:57.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:56 smithi064 ceph-mon[42500]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 763 op/s 2024-05-08T22:08:59.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:08:59 smithi064 ceph-mon[42500]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 711 op/s 2024-05-08T22:09:00.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:08:59 smithi160 ceph-mon[46962]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 711 op/s 2024-05-08T22:09:00.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:08:59 smithi186 ceph-mon[46619]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 711 op/s 2024-05-08T22:09:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 -- 172.21.15.64:0/1607899722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960104340 msgr2=0x7fe960106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 --2- 172.21.15.64:0/1607899722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960104340 0x7fe960106730 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe954009990 tx=0x7fe954009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 -- 172.21.15.64:0/1607899722 shutdown_connections 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 --2- 172.21.15.64:0/1607899722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960101a10 0x7fe960103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 --2- 172.21.15.64:0/1607899722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe960106c70 0x7fe960109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 --2- 172.21.15.64:0/1607899722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960104340 0x7fe960106730 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 -- 172.21.15.64:0/1607899722 >> 172.21.15.64:0/1607899722 conn(0x7fe9600fb3a0 msgr2=0x7fe9600fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.338+0000 7fe965512700 1 -- 172.21.15.64:0/1607899722 shutdown_connections 2024-05-08T22:09:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.339+0000 7fe965512700 1 -- 172.21.15.64:0/1607899722 wait complete. 2024-05-08T22:09:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 Processor -- start 2024-05-08T22:09:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 -- start start 2024-05-08T22:09:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960101a10 0x7fe960101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 0x7fe9600ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe960106c70 0x7fe9600fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9601006a0 con 0x7fe960101a10 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960100810 con 0x7fe960104340 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.340+0000 7fe965512700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9601a0ed0 con 0x7fe960106c70 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 0x7fe9600ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960101a10 0x7fe960101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 0x7fe9600ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:39180/0 (socket says 172.21.15.64:39180) 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960101a10 0x7fe960101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53754/0 (socket says 172.21.15.64:53754) 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 -- 172.21.15.64:0/450707655 learned_addr learned my addr 172.21.15.64:0/450707655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe960106c70 0x7fe9600fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 -- 172.21.15.64:0/450707655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe960106c70 msgr2=0x7fe9600fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe960106c70 0x7fe9600fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 -- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960101a10 msgr2=0x7fe960101330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960101a10 0x7fe960101330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 -- 172.21.15.64:0/450707655 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540095f0 con 0x7fe960104340 2024-05-08T22:09:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.341+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 0x7fe9600ff9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.353+0000 7fe947fff700 1 -- 172.21.15.64:0/450707655 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95401d070 con 0x7fe960104340 2024-05-08T22:09:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.353+0000 7fe947fff700 1 -- 172.21.15.64:0/450707655 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9540054f0 con 0x7fe960104340 2024-05-08T22:09:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.353+0000 7fe947fff700 1 -- 172.21.15.64:0/450707655 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954004c00 con 0x7fe960104340 2024-05-08T22:09:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.353+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9601a1100 con 0x7fe960104340 2024-05-08T22:09:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.353+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9601a1470 con 0x7fe960104340 2024-05-08T22:09:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.371+0000 7fe9716ccb80 1 Processor -- start 2024-05-08T22:09:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 -- start start 2024-05-08T22:09:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d0f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe947fff700 1 -- 172.21.15.64:0/450707655 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe95400fb90 con 0x7fe960104340 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f3ae0 0x56520d0f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d0410f0 con 0x56520d0f2bf0 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d03eab0 con 0x56520d0f2290 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.378+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d03a3a0 con 0x56520d0f3ae0 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53770/0 (socket says 172.21.15.64:53770) 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 -- 172.21.15.64:0/1440713639 learned_addr learned my addr 172.21.15.64:0/1440713639 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d0f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 -- 172.21.15.64:0/1440713639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f3ae0 msgr2=0x56520d0f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f3ae0 0x56520d0f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 -- 172.21.15.64:0/1440713639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 msgr2=0x56520d0f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d0f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 -- 172.21.15.64:0/1440713639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56520cebb350 con 0x56520d0f2bf0 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe947fff700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe9480a1270 0x7fe9480a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95effd700 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe950009d80 tx=0x7fe950011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5c252deda9aa17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe9480a1270 0x7fe9480a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.379+0000 7fe947fff700 1 -- 172.21.15.64:0/450707655 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe9540cd900 con 0x7fe960104340 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.380+0000 7fe9457fa700 1 -- 172.21.15.64:0/1440713639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950004230 con 0x56520d0f2bf0 2024-05-08T22:09:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.380+0000 7fe9457fa700 1 -- 172.21.15.64:0/1440713639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe950004390 con 0x56520d0f2bf0 2024-05-08T22:09:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.380+0000 7fe9457fa700 1 -- 172.21.15.64:0/1440713639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950019880 con 0x56520d0f2bf0 2024-05-08T22:09:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.381+0000 7fe9716ccb80 1 -- 172.21.15.64:0/1440713639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 msgr2=0x56520d0f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.381+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe950009d80 tx=0x7fe950011720 comp rx=0 tx=0).stop 2024-05-08T22:09:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.382+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe9480a1270 0x7fe9480a3730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe958009990 tx=0x7fe958009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.383+0000 7fe9716ccb80 1 -- 172.21.15.64:0/1440713639 shutdown_connections 2024-05-08T22:09:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.383+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d0f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.383+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f3ae0 0x56520d0f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.383+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/1440713639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3500 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.383+0000 7fe9716ccb80 1 -- 172.21.15.64:0/1440713639 >> 172.21.15.64:0/1440713639 conn(0x56520d0481b0 msgr2=0x56520d0e6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.386+0000 7fe9716ccb80 1 -- 172.21.15.64:0/1440713639 shutdown_connections 2024-05-08T22:09:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.387+0000 7fe9716ccb80 1 -- 172.21.15.64:0/1440713639 wait complete. 2024-05-08T22:09:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 -- start start 2024-05-08T22:09:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 Processor -- start 2024-05-08T22:09:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2290 0x56520d0eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d0eb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520cfd6500 con 0x56520d0f2290 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520cfd61a0 con 0x56520d0f3ae0 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520cebd640 con 0x56520d0f2bf0 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.388+0000 7fe9716ccb80 10 client.0 _ll_register_callbacks cb 0x56520cfdfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d0eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:39204/0 (socket says 172.21.15.64:39204) 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d0eb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:43810/0 (socket says 172.21.15.64:43810) 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 -- 172.21.15.64:0/2310080908 learned_addr learned my addr 172.21.15.64:0/2310080908 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 -- 172.21.15.64:0/2310080908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 msgr2=0x56520d0eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d0eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 -- 172.21.15.64:0/2310080908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2290 msgr2=0x56520d0eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2290 0x56520d0eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 -- 172.21.15.64:0/2310080908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9500099e0 con 0x56520d0f3ae0 2024-05-08T22:09:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d0eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f0590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe958007780 tx=0x7fe958004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb2cdb0e8f1cccd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe942ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe942ff5700 1 -- 172.21.15.64:0/2310080908 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958023640 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe942ff5700 1 -- 172.21.15.64:0/2310080908 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe9580237a0 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.389+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56520cf93210 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe942ff5700 1 -- 172.21.15.64:0/2310080908 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe958023960 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 10 client.26000 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56520ce67b00 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe942ff5700 1 -- 172.21.15.64:0/2310080908 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56520cf93210 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe942ff5700 1 -- 172.21.15.64:0/2310080908 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe958016a90 con 0x56520d0f3ae0 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 10 client.26000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 20 client.26000 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d110840 0x56520d112c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 4 client.26000 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.390+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56520d0b97b0 con 0x56520d110840 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.392+0000 7fe95effd700 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d110840 0x56520d112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.393+0000 7fe942ff5700 10 client.26000 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.393+0000 7fe95effd700 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d110840 0x56520d112c60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1725da410fd64bc server_cookie=b2afdead25cba97c in_seq=0 out_seq=0 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.393+0000 7fe942ff5700 1 -- 172.21.15.64:0/2310080908 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56520d0b97b0 con 0x56520d110840 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.393+0000 7fe942ff5700 10 client.26000 handle_command_reply: tid=0 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.393+0000 7fe9716ccb80 1 client.26000 shutdown 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.393+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 msgr2=0x56520d0f0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f0590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe958007780 tx=0x7fe958004ad0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 shutdown_connections 2024-05-08T22:09:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f0590 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d110840 0x56520d112c60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2290 0x56520d0eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2310080908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d0eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 >> 172.21.15.64:0/2310080908 conn(0x56520d0481b0 msgr2=0x56520d0e4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe942ff5700 0 client.26000 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 shutdown_connections 2024-05-08T22:09:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.394+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2310080908 wait complete. 2024-05-08T22:09:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.398+0000 7fe9716ccb80 20 client.26000 trim_cache size 0 max 16384 2024-05-08T22:09:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.417+0000 7fe9716ccb80 1 Processor -- start 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 -- start start 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2290 0x56520d0f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d0f09f0 con 0x56520d0f2bf0 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d0f1e30 con 0x56520d0f3ae0 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d0f32f0 con 0x56520d0f2290 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53788/0 (socket says 172.21.15.64:53788) 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 -- 172.21.15.64:0/3690817978 learned_addr learned my addr 172.21.15.64:0/3690817978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95effd700 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2290 0x56520d0f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95f7fe700 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 -- 172.21.15.64:0/3690817978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2290 msgr2=0x56520d0f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2290 0x56520d0f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 -- 172.21.15.64:0/3690817978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 msgr2=0x56520d0f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.418+0000 7fe95e7fc700 1 -- 172.21.15.64:0/3690817978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56520d000500 con 0x56520d0f2bf0 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.419+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe954097860 tx=0x7fe954009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75acbbbce801bc95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.419+0000 7fe945ffb700 1 -- 172.21.15.64:0/3690817978 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95409a750 con 0x56520d0f2bf0 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.419+0000 7fe945ffb700 1 -- 172.21.15.64:0/3690817978 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe954016e80 con 0x56520d0f2bf0 2024-05-08T22:09:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.419+0000 7fe945ffb700 1 -- 172.21.15.64:0/3690817978 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954099190 con 0x56520d0f2bf0 2024-05-08T22:09:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 -- 172.21.15.64:0/3690817978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 msgr2=0x56520d0f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe954097860 tx=0x7fe954009310 comp rx=0 tx=0).stop 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 -- 172.21.15.64:0/3690817978 shutdown_connections 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f3ae0 0x56520d0f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2290 0x56520d0f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/3690817978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f2bf0 0x56520d0f3f20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 -- 172.21.15.64:0/3690817978 >> 172.21.15.64:0/3690817978 conn(0x56520d0481b0 msgr2=0x56520d0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 -- 172.21.15.64:0/3690817978 shutdown_connections 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 -- 172.21.15.64:0/3690817978 wait complete. 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 -- start start 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.420+0000 7fe9716ccb80 1 Processor -- start 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d121010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d11fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f3ae0 0x56520d120150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d120860 con 0x56520d0f3ae0 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d1209d0 con 0x56520d0f2290 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520d165810 con 0x56520d0f2bf0 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe9716ccb80 10 client.0 _ll_register_callbacks cb 0x56520cfdfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d11fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d11fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:43834/0 (socket says 172.21.15.64:43834) 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 -- 172.21.15.64:0/2878113218 learned_addr learned my addr 172.21.15.64:0/2878113218 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 -- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 msgr2=0x56520d121010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d121010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 -- 172.21.15.64:0/2878113218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f3ae0 msgr2=0x56520d120150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f3ae0 0x56520d120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.421+0000 7fe95e7fc700 1 -- 172.21.15.64:0/2878113218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9601a1470 con 0x56520d0f2bf0 2024-05-08T22:09:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.422+0000 7fe95e7fc700 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d11fb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe9540095c0 tx=0x7fe954005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3914d6e729ff5345 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.422+0000 7fe942ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.422+0000 7fe942ff5700 1 -- 172.21.15.64:0/2878113218 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95409adf0 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.422+0000 7fe942ff5700 1 -- 172.21.15.64:0/2878113218 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe95409ec20 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.422+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe950004390 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe942ff5700 1 -- 172.21.15.64:0/2878113218 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe95409b4f0 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 10 client.25849 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56520cebfc70 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe942ff5700 1 -- 172.21.15.64:0/2878113218 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9540b8070 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe942ff5700 1 -- 172.21.15.64:0/2878113218 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe954099d50 con 0x56520d0f2bf0 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 10 client.25849 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 20 client.25849 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d166d20 0x56520d169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 4 client.25849 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:09:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56520d032e60 con 0x56520d166d20 2024-05-08T22:09:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.423+0000 7fe95effd700 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d166d20 0x56520d169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.424+0000 7fe942ff5700 10 client.25849 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.424+0000 7fe95effd700 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d166d20 0x56520d169140 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ed51daf57f2aeb server_cookie=b83dffc4fbd0881f in_seq=0 out_seq=0 2024-05-08T22:09:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe942ff5700 1 -- 172.21.15.64:0/2878113218 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56520d032e60 con 0x56520d166d20 2024-05-08T22:09:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe942ff5700 10 client.25849 handle_command_reply: tid=0 2024-05-08T22:09:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 client.25849 shutdown 2024-05-08T22:09:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 msgr2=0x56520d11fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d11fb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe9540095c0 tx=0x7fe954005fd0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 shutdown_connections 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56520d0f2290 0x56520d121010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x56520d166d20 0x56520d169140 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56520d0f3ae0 0x56520d120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 --2- 172.21.15.64:0/2878113218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56520d0f2bf0 0x56520d11fb40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 >> 172.21.15.64:0/2878113218 conn(0x56520d0481b0 msgr2=0x56520d0ea530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe942ff5700 0 client.25849 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 shutdown_connections 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 1 -- 172.21.15.64:0/2878113218 wait complete. 2024-05-08T22:09:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.432+0000 7fe9716ccb80 20 client.25849 trim_cache size 0 max 16384 2024-05-08T22:09:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:09:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:09:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:09:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe9480a1270 msgr2=0x7fe9480a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe9480a1270 0x7fe9480a3730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe958009990 tx=0x7fe958009130 comp rx=0 tx=0).stop 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 msgr2=0x7fe9600ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 0x7fe9600ff9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954004730 comp rx=0 tx=0).stop 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 shutdown_connections 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fe9480a1270 0x7fe9480a3730 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe960106c70 0x7fe9600fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe960101a10 0x7fe960101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 --2- 172.21.15.64:0/450707655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960104340 0x7fe9600ff9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 >> 172.21.15.64:0/450707655 conn(0x7fe9600fb3a0 msgr2=0x7fe960102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 shutdown_connections 2024-05-08T22:09:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:00.433+0000 7fe965512700 1 -- 172.21.15.64:0/450707655 wait complete. 2024-05-08T22:09:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 850.0 s, 1300.6 tps, lat 76.880 ms stddev 93.447 2024-05-08T22:09:01.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:01 smithi064 ceph-mon[42500]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 750 op/s 2024-05-08T22:09:02.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:01 smithi160 ceph-mon[46962]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 750 op/s 2024-05-08T22:09:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:01 smithi186 ceph-mon[46619]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 750 op/s 2024-05-08T22:09:03.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:03 smithi064 ceph-mon[42500]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 786 op/s 2024-05-08T22:09:04.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:03 smithi160 ceph-mon[46962]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 786 op/s 2024-05-08T22:09:04.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:03 smithi186 ceph-mon[46619]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 786 op/s 2024-05-08T22:09:06.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:05 smithi160 ceph-mon[46962]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 775 op/s 2024-05-08T22:09:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:05 smithi186 ceph-mon[46619]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 775 op/s 2024-05-08T22:09:06.215 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:05 smithi064 ceph-mon[42500]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 775 op/s 2024-05-08T22:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 855.0 s, 1393.0 tps, lat 71.569 ms stddev 76.359 2024-05-08T22:09:08.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:07 smithi160 ceph-mon[46962]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 760 op/s 2024-05-08T22:09:08.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:07 smithi186 ceph-mon[46619]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 760 op/s 2024-05-08T22:09:08.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:07 smithi064 ceph-mon[42500]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 760 op/s 2024-05-08T22:09:10.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:09 smithi160 ceph-mon[46962]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 759 op/s 2024-05-08T22:09:10.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:09 smithi186 ceph-mon[46619]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 759 op/s 2024-05-08T22:09:10.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:09 smithi064 ceph-mon[42500]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 759 op/s 2024-05-08T22:09:11.477 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 860.0 s, 1529.5 tps, lat 65.275 ms stddev 70.227 2024-05-08T22:09:12.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:11 smithi160 ceph-mon[46962]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 757 op/s 2024-05-08T22:09:12.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:11 smithi186 ceph-mon[46619]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 757 op/s 2024-05-08T22:09:12.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:11 smithi064 ceph-mon[42500]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 757 op/s 2024-05-08T22:09:13.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:12 smithi186 ceph-mon[46619]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 751 op/s 2024-05-08T22:09:13.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:12 smithi064 ceph-mon[42500]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 751 op/s 2024-05-08T22:09:13.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:12 smithi160 ceph-mon[46962]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 751 op/s 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.601+0000 7f0900d13700 1 -- 172.21.15.64:0/427005621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 msgr2=0x7f08fc1093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.601+0000 7f0900d13700 1 --2- 172.21.15.64:0/427005621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc1093e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f08f400b0a0 tx=0x7f08f400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 -- 172.21.15.64:0/427005621 shutdown_connections 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 --2- 172.21.15.64:0/427005621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08fc0ff1c0 0x7f08fc0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 --2- 172.21.15.64:0/427005621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc1093e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 --2- 172.21.15.64:0/427005621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 0x7f08fc1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 -- 172.21.15.64:0/427005621 >> 172.21.15.64:0/427005621 conn(0x7f08fc0faaf0 msgr2=0x7f08fc0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 -- 172.21.15.64:0/427005621 shutdown_connections 2024-05-08T22:09:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.602+0000 7f0900d13700 1 -- 172.21.15.64:0/427005621 wait complete. 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 Processor -- start 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 -- start start 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08fc0ff1c0 0x7f08fc101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 0x7f08fc100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc19ae00 con 0x7f08fc102b30 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc100bc0 con 0x7f08fc0ff1c0 2024-05-08T22:09:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.603+0000 7f0900d13700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc100d80 con 0x7f08fc102190 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53674/0 (socket says 172.21.15.64:53674) 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 -- 172.21.15.64:0/550444597 learned_addr learned my addr 172.21.15.64:0/550444597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 0x7f08fc100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fa59c700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08fc0ff1c0 0x7f08fc101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 -- 172.21.15.64:0/550444597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 msgr2=0x7f08fc100140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 0x7f08fc100140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 -- 172.21.15.64:0/550444597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08fc0ff1c0 msgr2=0x7f08fc101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08fc0ff1c0 0x7f08fc101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08fad9d700 1 -- 172.21.15.64:0/550444597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009d00 con 0x7f08fc102b30 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.604+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 0x7f08fc100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.605+0000 7f08fad9d700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc100680 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009030 tx=0x7f08f400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.605+0000 7f08eb7fe700 1 -- 172.21.15.64:0/550444597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f400e040 con 0x7f08fc102b30 2024-05-08T22:09:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.605+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc100fa0 con 0x7f08fc102b30 2024-05-08T22:09:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.605+0000 7f08eb7fe700 1 -- 172.21.15.64:0/550444597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f4004e60 con 0x7f08fc102b30 2024-05-08T22:09:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.605+0000 7f08eb7fe700 1 -- 172.21.15.64:0/550444597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4005370 con 0x7f08fc102b30 2024-05-08T22:09:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.605+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc19b2b0 con 0x7f08fc102b30 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.609+0000 7f090cecdb80 1 Processor -- start 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.609+0000 7f08eb7fe700 1 -- 172.21.15.64:0/550444597 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f08f40054d0 con 0x7f08fc102b30 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.609+0000 7f090cecdb80 1 -- start start 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f090cecdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b5fd0f0 con 0x55656b6a63e0 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f090cecdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b5faab0 con 0x55656b6a72d0 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f090cecdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b5f63a0 con 0x55656b6a5a80 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48908/0 (socket says 172.21.15.64:48908) 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 -- 172.21.15.64:0/1085618346 learned_addr learned my addr 172.21.15.64:0/1085618346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08eb7fe700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f08e40a1120 0x7f08e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 -- 172.21.15.64:0/1085618346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 msgr2=0x55656b6a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 -- 172.21.15.64:0/1085618346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 msgr2=0x55656b6a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08f9d9b700 1 -- 172.21.15.64:0/1085618346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656b477350 con 0x55656b6a5a80 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08fad9d700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f08e40a1120 0x7f08e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.610+0000 7f08eb7fe700 1 -- 172.21.15.64:0/550444597 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08f40cdf30 con 0x7f08fc102b30 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.611+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6a5ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009ee0 tx=0x7f08f0009a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270421016415fa41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.611+0000 7f08fad9d700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f08e40a1120 0x7f08e40a35e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009fd0 tx=0x7f08f4090bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.611+0000 7f08e8ff9700 1 -- 172.21.15.64:0/1085618346 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f001a070 con 0x55656b6a5a80 2024-05-08T22:09:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.611+0000 7f08e8ff9700 1 -- 172.21.15.64:0/1085618346 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f0014170 con 0x55656b6a5a80 2024-05-08T22:09:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.611+0000 7f08e8ff9700 1 -- 172.21.15.64:0/1085618346 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f00106a0 con 0x55656b6a5a80 2024-05-08T22:09:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 -- 172.21.15.64:0/1085618346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 msgr2=0x55656b6a5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6a5ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009ee0 tx=0x7f08f0009a40 comp rx=0 tx=0).stop 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 -- 172.21.15.64:0/1085618346 shutdown_connections 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6a5ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1085618346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.612+0000 7f090cecdb80 1 -- 172.21.15.64:0/1085618346 >> 172.21.15.64:0/1085618346 conn(0x55656b6041b0 msgr2=0x55656b6a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.614+0000 7f090cecdb80 1 -- 172.21.15.64:0/1085618346 shutdown_connections 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.614+0000 7f090cecdb80 1 -- 172.21.15.64:0/1085618346 wait complete. 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.614+0000 7f090cecdb80 1 -- start start 2024-05-08T22:09:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.614+0000 7f090cecdb80 1 Processor -- start 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b592500 con 0x55656b6a63e0 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b5921a0 con 0x55656b6a72d0 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b479640 con 0x55656b6a5a80 2024-05-08T22:09:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f090cecdb80 10 client.0 _ll_register_callbacks cb 0x55656b59bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08fa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08fa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53690/0 (socket says 172.21.15.64:53690) 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08fa59c700 1 -- 172.21.15.64:0/1141805359 learned_addr learned my addr 172.21.15.64:0/1141805359 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08fa59c700 1 -- 172.21.15.64:0/1141805359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 msgr2=0x55656b6af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.615+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08fa59c700 1 -- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 msgr2=0x55656b6b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08fa59c700 1 -- 172.21.15.64:0/1141805359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f0014470 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6afd90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f08ec010670 tx=0x7f08ec010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959c2dca2a96410b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08e1ffb700 1 -- 172.21.15.64:0/1141805359 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec012860 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08e1ffb700 1 -- 172.21.15.64:0/1141805359 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08ec0129c0 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55656b54f210 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08e1ffb700 1 -- 172.21.15.64:0/1141805359 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08ec021030 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f090cecdb80 10 client.16221 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55656b423b00 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.616+0000 7f08e1ffb700 1 -- 172.21.15.64:0/1141805359 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55656b54f210 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f08e1ffb700 1 -- 172.21.15.64:0/1141805359 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f08ec0263f0 con 0x55656b6a63e0 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f090cecdb80 10 client.16221 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f090cecdb80 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f090cecdb80 20 client.16221 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b6bb6b0 0x55656b6bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f090cecdb80 4 client.16221 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55656b6757b0 con 0x55656b6bb6b0 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b6bb6b0 0x55656b6bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b6bb6b0 0x55656b6bdad0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53efe90b879b9c85 server_cookie=9cf612d04ac48e51 in_seq=0 out_seq=0 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.617+0000 7f08e1ffb700 10 client.16221 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f08e1ffb700 1 -- 172.21.15.64:0/1141805359 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55656b6757b0 con 0x55656b6bb6b0 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f08e1ffb700 10 client.16221 handle_command_reply: tid=0 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 client.16221 shutdown 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 msgr2=0x55656b6afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6afd90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f08ec010670 tx=0x7f08ec010980 comp rx=0 tx=0).stop 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 shutdown_connections 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b6bb6b0 0x55656b6bdad0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a5a80 0x55656b6af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1141805359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a63e0 0x55656b6afd90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f08e1ffb700 0 client.16221 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 >> 172.21.15.64:0/1141805359 conn(0x55656b6041b0 msgr2=0x55656b6a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 shutdown_connections 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.618+0000 7f090cecdb80 1 -- 172.21.15.64:0/1141805359 wait complete. 2024-05-08T22:09:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.619+0000 7f090cecdb80 20 client.16221 trim_cache size 0 max 16384 2024-05-08T22:09:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 Processor -- start 2024-05-08T22:09:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 -- start start 2024-05-08T22:09:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6bdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b6af3f0 con 0x55656b6a5a80 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b6c2cc0 con 0x55656b6a72d0 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.640+0000 7f090cecdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b6ad4b0 con 0x55656b6a63e0 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:48918/0 (socket says 172.21.15.64:48918) 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 -- 172.21.15.64:0/1339022321 learned_addr learned my addr 172.21.15.64:0/1339022321 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6bdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 -- 172.21.15.64:0/1339022321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 msgr2=0x55656b6b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 -- 172.21.15.64:0/1339022321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 msgr2=0x55656b6bdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6bdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 -- 172.21.15.64:0/1339022321 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656b5bc500 con 0x55656b6a63e0 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.641+0000 7f08fa59c700 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6a5e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097d0 tx=0x7f08ec010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8168543ecb46a5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.642+0000 7f08e97fa700 1 -- 172.21.15.64:0/1339022321 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec021040 con 0x55656b6a63e0 2024-05-08T22:09:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.642+0000 7f08e97fa700 1 -- 172.21.15.64:0/1339022321 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08ec026570 con 0x55656b6a63e0 2024-05-08T22:09:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.642+0000 7f08e97fa700 1 -- 172.21.15.64:0/1339022321 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004910 con 0x55656b6a63e0 2024-05-08T22:09:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.642+0000 7f090cecdb80 1 -- 172.21.15.64:0/1339022321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 msgr2=0x55656b6a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.642+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6a5e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097d0 tx=0x7f08ec010d80 comp rx=0 tx=0).stop 2024-05-08T22:09:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 -- 172.21.15.64:0/1339022321 shutdown_connections 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6bdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 --2- 172.21.15.64:0/1339022321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6a5e60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 -- 172.21.15.64:0/1339022321 >> 172.21.15.64:0/1339022321 conn(0x55656b6041b0 msgr2=0x55656b6aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 -- 172.21.15.64:0/1339022321 shutdown_connections 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 -- 172.21.15.64:0/1339022321 wait complete. 2024-05-08T22:09:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.645+0000 7f090cecdb80 1 -- start start 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 Processor -- start 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b6b5de0 con 0x55656b6a5a80 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b6b5f50 con 0x55656b6a72d0 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656b6b6110 con 0x55656b6a63e0 2024-05-08T22:09:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f090cecdb80 10 client.0 _ll_register_callbacks cb 0x55656b59bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f08f9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f08f9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53702/0 (socket says 172.21.15.64:53702) 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f08f9d9b700 1 -- 172.21.15.64:0/2282839988 learned_addr learned my addr 172.21.15.64:0/2282839988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.646+0000 7f08fa59c700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08f9d9b700 1 -- 172.21.15.64:0/2282839988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 msgr2=0x55656b6b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08f9d9b700 1 -- 172.21.15.64:0/2282839988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 msgr2=0x55656b6b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08f9d9b700 1 -- 172.21.15.64:0/2282839988 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55656b423b00 con 0x55656b6a72d0 2024-05-08T22:09:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08f9d9b700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b58a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009780 tx=0x7f08f0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3249973db1aae728 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08e1ffb700 1 -- 172.21.15.64:0/2282839988 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f001a040 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08e1ffb700 1 -- 172.21.15.64:0/2282839988 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08f002c600 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08f0014170 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f08e1ffb700 1 -- 172.21.15.64:0/2282839988 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08f002c7c0 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f090cecdb80 10 client.26018 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.647+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55656b47bc70 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f08e1ffb700 1 -- 172.21.15.64:0/2282839988 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f003d020 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f08e1ffb700 1 -- 172.21.15.64:0/2282839988 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f08f00118c0 con 0x55656b6a72d0 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f090cecdb80 10 client.26018 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f090cecdb80 10 client.26018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f090cecdb80 20 client.26018 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f090cecdb80 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b7121f0 0x55656b714610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f090cecdb80 4 client.26018 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:09:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55656b5eee60 con 0x55656b7121f0 2024-05-08T22:09:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f08fa59c700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b7121f0 0x55656b714610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f08e1ffb700 10 client.26018 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.648+0000 7f08fa59c700 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b7121f0 0x55656b714610 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335d92d83b31f412 server_cookie=4d622f951cbc560b in_seq=0 out_seq=0 2024-05-08T22:09:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f08e1ffb700 1 -- 172.21.15.64:0/2282839988 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55656b5eee60 con 0x55656b7121f0 2024-05-08T22:09:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f08e1ffb700 10 client.26018 handle_command_reply: tid=0 2024-05-08T22:09:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 client.26018 shutdown 2024-05-08T22:09:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 msgr2=0x55656b6b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b58a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009780 tx=0x7f08f0005880 comp rx=0 tx=0).stop 2024-05-08T22:09:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 shutdown_connections 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55656b6a72d0 0x55656b6b58a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55656b7121f0 0x55656b714610 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55656b6a5a80 0x55656b6b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f08e1ffb700 0 client.26018 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 --2- 172.21.15.64:0/2282839988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55656b6a63e0 0x55656b6b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.655+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 >> 172.21.15.64:0/2282839988 conn(0x55656b6041b0 msgr2=0x55656b6aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 shutdown_connections 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f090cecdb80 1 -- 172.21.15.64:0/2282839988 wait complete. 2024-05-08T22:09:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f090cecdb80 20 client.26018 trim_cache size 0 max 16384 2024-05-08T22:09:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:09:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:09:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:09:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:09:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f08e40a1120 msgr2=0x7f08e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f08e40a1120 0x7f08e40a35e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009fd0 tx=0x7f08f4090bc0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 msgr2=0x7f08fc100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc100680 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009030 tx=0x7f08f400bb60 comp rx=0 tx=0).stop 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 shutdown_connections 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f08e40a1120 0x7f08e40a35e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08fc0ff1c0 0x7f08fc101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08fc102b30 0x7f08fc100680 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 --2- 172.21.15.64:0/550444597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08fc102190 0x7f08fc100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.656+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 >> 172.21.15.64:0/550444597 conn(0x7f08fc0faaf0 msgr2=0x7f08fc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.657+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 shutdown_connections 2024-05-08T22:09:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:13.657+0000 7f0900d13700 1 -- 172.21.15.64:0/550444597 wait complete. 2024-05-08T22:09:16.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:15 smithi160 ceph-mon[46962]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 699 op/s 2024-05-08T22:09:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:15 smithi186 ceph-mon[46619]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 699 op/s 2024-05-08T22:09:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:15 smithi064 ceph-mon[42500]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 699 op/s 2024-05-08T22:09:16.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 865.0 s, 1247.8 tps, lat 80.584 ms stddev 88.014 2024-05-08T22:09:18.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:17 smithi160 ceph-mon[46962]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 741 op/s 2024-05-08T22:09:18.148 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:17 smithi186 ceph-mon[46619]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 741 op/s 2024-05-08T22:09:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:17 smithi064 ceph-mon[42500]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 741 op/s 2024-05-08T22:09:19.047 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:18 smithi160 ceph-mon[46962]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-05-08T22:09:19.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:18 smithi186 ceph-mon[46619]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-05-08T22:09:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:18 smithi064 ceph-mon[42500]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-05-08T22:09:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 870.0 s, 969.0 tps, lat 103.062 ms stddev 203.117 2024-05-08T22:09:22.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:21 smithi160 ceph-mon[46962]: pgmap v701: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 23 MiB/s wr, 1.42k op/s 2024-05-08T22:09:22.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:21 smithi186 ceph-mon[46619]: pgmap v701: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 23 MiB/s wr, 1.42k op/s 2024-05-08T22:09:22.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:21 smithi064 ceph-mon[42500]: pgmap v701: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 23 MiB/s wr, 1.42k op/s 2024-05-08T22:09:24.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:23 smithi160 ceph-mon[46962]: pgmap v702: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-05-08T22:09:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:23 smithi186 ceph-mon[46619]: pgmap v702: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-05-08T22:09:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:23 smithi064 ceph-mon[42500]: pgmap v702: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-05-08T22:09:26.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:25 smithi160 ceph-mon[46962]: pgmap v703: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.88k op/s 2024-05-08T22:09:26.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:25 smithi186 ceph-mon[46619]: pgmap v703: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.88k op/s 2024-05-08T22:09:26.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:25 smithi064 ceph-mon[42500]: pgmap v703: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.88k op/s 2024-05-08T22:09:26.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 875.0 s, 1413.4 tps, lat 71.171 ms stddev 85.084 2024-05-08T22:09:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 -- 172.21.15.64:0/4175340083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba8104ab0 msgr2=0x7f7ba8104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/4175340083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba8104ab0 0x7f7ba8104f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009f90 tx=0x7f7ba000b330 comp rx=0 tx=0).stop 2024-05-08T22:09:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 -- 172.21.15.64:0/4175340083 shutdown_connections 2024-05-08T22:09:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/4175340083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba8104ab0 0x7f7ba8104f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/4175340083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 0x7f7ba8101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/4175340083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba81020e0 0x7f7ba8104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 -- 172.21.15.64:0/4175340083 >> 172.21.15.64:0/4175340083 conn(0x7f7ba80fb390 msgr2=0x7f7ba80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 -- 172.21.15.64:0/4175340083 shutdown_connections 2024-05-08T22:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.842+0000 7f7baf6aa700 1 -- 172.21.15.64:0/4175340083 wait complete. 2024-05-08T22:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 Processor -- start 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 -- start start 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 0x7f7ba819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 0x7f7ba819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba8104ab0 0x7f7ba81a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81a5090 con 0x7f7ba80ff770 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81a51d0 con 0x7f7ba81020e0 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7baf6aa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81a5360 con 0x7f7ba8104ab0 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7bad446700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 0x7f7ba819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 0x7f7ba819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7badc47700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba8104ab0 0x7f7ba81a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7bad446700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 0x7f7ba819d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58918/0 (socket says 172.21.15.64:58918) 2024-05-08T22:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 0x7f7ba819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40162/0 (socket says 172.21.15.64:40162) 2024-05-08T22:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.843+0000 7f7bad446700 1 -- 172.21.15.64:0/810445802 learned_addr learned my addr 172.21.15.64:0/810445802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7bacc45700 1 -- 172.21.15.64:0/810445802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba8104ab0 msgr2=0x7f7ba81a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7bacc45700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba8104ab0 0x7f7ba81a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7bacc45700 1 -- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 msgr2=0x7f7ba819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7bacc45700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 0x7f7ba819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7bacc45700 1 -- 172.21.15.64:0/810445802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40095f0 con 0x7f7ba81020e0 2024-05-08T22:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7bacc45700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 0x7f7ba819daa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800e9f0 tx=0x7f7b9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7b9e7fc700 1 -- 172.21.15.64:0/810445802 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800c9c0 con 0x7f7ba81020e0 2024-05-08T22:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7b9e7fc700 1 -- 172.21.15.64:0/810445802 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b98010430 con 0x7f7ba81020e0 2024-05-08T22:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.844+0000 7f7b9e7fc700 1 -- 172.21.15.64:0/810445802 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98018680 con 0x7f7ba81020e0 2024-05-08T22:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.845+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba0009c40 con 0x7f7ba81020e0 2024-05-08T22:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.846+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba81a5940 con 0x7f7ba81020e0 2024-05-08T22:09:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.848+0000 7f7b9e7fc700 1 -- 172.21.15.64:0/810445802 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7b980105a0 con 0x7f7ba81020e0 2024-05-08T22:09:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.849+0000 7f7b9e7fc700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b940a12f0 0x7f7b940a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.849+0000 7f7b9e7fc700 1 -- 172.21.15.64:0/810445802 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b98014070 con 0x7f7ba81020e0 2024-05-08T22:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.849+0000 7f7bbb864b80 1 Processor -- start 2024-05-08T22:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.849+0000 7f7bad446700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b940a12f0 0x7f7b940a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 -- start start 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bad446700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b940a12f0 0x7f7b940a37b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4005d10 tx=0x7f7ba4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808290f0 con 0x5598808d23e0 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559880826ab0 con 0x5598808d32d0 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808223a0 con 0x5598808d1a80 2024-05-08T22:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58936/0 (socket says 172.21.15.64:58936) 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bacc45700 1 -- 172.21.15.64:0/729551071 learned_addr learned my addr 172.21.15.64:0/729551071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7bad446700 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.850+0000 7f7badc47700 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7badc47700 1 -- 172.21.15.64:0/729551071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 msgr2=0x5598808d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7badc47700 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808d1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7badc47700 1 -- 172.21.15.64:0/729551071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 msgr2=0x5598808d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7badc47700 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7badc47700 1 -- 172.21.15.64:0/729551071 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598806a3350 con 0x5598808d32d0 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7badc47700 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808d3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0007900 tx=0x7f7ba0009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3505e8355b11d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7b937fe700 1 -- 172.21.15.64:0/729551071 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000e040 con 0x5598808d32d0 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7b937fe700 1 -- 172.21.15.64:0/729551071 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba0007620 con 0x5598808d32d0 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.851+0000 7f7b937fe700 1 -- 172.21.15.64:0/729551071 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000bd40 con 0x5598808d32d0 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.852+0000 7f7bbb864b80 1 -- 172.21.15.64:0/729551071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 msgr2=0x5598808d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.852+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808d3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0007900 tx=0x7f7ba0009ba0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 -- 172.21.15.64:0/729551071 shutdown_connections 2024-05-08T22:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808d3750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/729551071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 -- 172.21.15.64:0/729551071 >> 172.21.15.64:0/729551071 conn(0x5598808301b0 msgr2=0x5598808ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 -- 172.21.15.64:0/729551071 shutdown_connections 2024-05-08T22:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 -- 172.21.15.64:0/729551071 wait complete. 2024-05-08T22:09:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 -- start start 2024-05-08T22:09:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.853+0000 7f7bbb864b80 1 Processor -- start 2024-05-08T22:09:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598807be500 con 0x5598808d23e0 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598807be1a0 con 0x5598808d32d0 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598806a5640 con 0x5598808d1a80 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bbb864b80 10 client.0 _ll_register_callbacks cb 0x5598807c7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808ecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58944/0 (socket says 172.21.15.64:58944) 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 -- 172.21.15.64:0/3756009073 learned_addr learned my addr 172.21.15.64:0/3756009073 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7badc47700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bad446700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 -- 172.21.15.64:0/3756009073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 msgr2=0x5598808ec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808ec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 -- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 msgr2=0x5598808f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bacc45700 1 -- 172.21.15.64:0/3756009073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8105a60 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.854+0000 7f7bad446700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7bacc45700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808ecd60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c5b0 tx=0x7f7b9809a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c38db5300ea73f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7b90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7b90ff9700 1 -- 172.21.15.64:0/3756009073 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9809dd10 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7b90ff9700 1 -- 172.21.15.64:0/3756009073 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b980a2360 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55988077b210 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7b90ff9700 1 -- 172.21.15.64:0/3756009073 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b980993d0 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7bbb864b80 10 client.16239 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.855+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55988064fb00 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7b90ff9700 1 -- 172.21.15.64:0/3756009073 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55988077b210 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7b90ff9700 1 -- 172.21.15.64:0/3756009073 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b98094af0 con 0x5598808d23e0 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bbb864b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bbb864b80 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bbb864b80 20 client.16239 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880909710 0x55988090bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bbb864b80 4 client.16239 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598808a17b0 con 0x559880909710 2024-05-08T22:09:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bad446700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880909710 0x55988090bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7bad446700 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880909710 0x55988090bb30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589a80a9ac8190a0 server_cookie=af9d798a58b9c2dd in_seq=0 out_seq=0 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.856+0000 7f7b90ff9700 10 client.16239 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7b90ff9700 1 -- 172.21.15.64:0/3756009073 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598808a17b0 con 0x559880909710 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7b90ff9700 10 client.16239 handle_command_reply: tid=0 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 client.16239 shutdown 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 msgr2=0x5598808ecd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808ecd60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c5b0 tx=0x7f7b9809a5e0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 shutdown_connections 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d32d0 0x5598808f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880909710 0x55988090bb30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d1a80 0x5598808ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3756009073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d23e0 0x5598808ecd60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7b90ff9700 0 client.16239 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 >> 172.21.15.64:0/3756009073 conn(0x5598808301b0 msgr2=0x5598808cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 shutdown_connections 2024-05-08T22:09:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.857+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3756009073 wait complete. 2024-05-08T22:09:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.858+0000 7f7bbb864b80 20 client.16239 trim_cache size 0 max 16384 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 Processor -- start 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 -- start start 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808ed720 con 0x5598808d1a80 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808cd770 con 0x5598808d23e0 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.876+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559880905f30 con 0x5598808d32d0 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7badc47700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58960/0 (socket says 172.21.15.64:58960) 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 -- 172.21.15.64:0/3183420792 learned_addr learned my addr 172.21.15.64:0/3183420792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bacc45700 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 -- 172.21.15.64:0/3183420792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 msgr2=0x5598808cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 -- 172.21.15.64:0/3183420792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 msgr2=0x5598808f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 -- 172.21.15.64:0/3183420792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598807e8500 con 0x5598808d1a80 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.877+0000 7f7bad446700 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808cc910 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55988090c400 tx=0x7f7ba4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b5892dc3affb5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.878+0000 7f7b93fff700 1 -- 172.21.15.64:0/3183420792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4023040 con 0x5598808d1a80 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.878+0000 7f7b93fff700 1 -- 172.21.15.64:0/3183420792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba400e3b0 con 0x5598808d1a80 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.878+0000 7f7b93fff700 1 -- 172.21.15.64:0/3183420792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4003a70 con 0x5598808d1a80 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3183420792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 msgr2=0x5598808cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808cc910 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55988090c400 tx=0x7f7ba4017000 comp rx=0 tx=0).stop 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3183420792 shutdown_connections 2024-05-08T22:09:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808cc910 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/3183420792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3183420792 >> 172.21.15.64:0/3183420792 conn(0x5598808301b0 msgr2=0x5598808ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3183420792 shutdown_connections 2024-05-08T22:09:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.879+0000 7f7bbb864b80 1 -- 172.21.15.64:0/3183420792 wait complete. 2024-05-08T22:09:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 -- start start 2024-05-08T22:09:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 Processor -- start 2024-05-08T22:09:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808f40a0 con 0x5598808d1a80 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808f2960 con 0x5598808d23e0 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.880+0000 7f7bbb864b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598808f2b20 con 0x5598808d32d0 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bbb864b80 10 client.0 _ll_register_callbacks cb 0x5598807c7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bad446700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:40188/0 (socket says 172.21.15.64:40188) 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 -- 172.21.15.64:0/4149562471 learned_addr learned my addr 172.21.15.64:0/4149562471 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7badc47700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 -- 172.21.15.64:0/4149562471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 msgr2=0x5598808f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 -- 172.21.15.64:0/4149562471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 msgr2=0x5598808f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 -- 172.21.15.64:0/4149562471 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba4017070 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7badc47700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bacc45700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f1ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b98094e20 tx=0x7f7b980962f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41a03b99f706a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7b90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7b90ff9700 1 -- 172.21.15.64:0/4149562471 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b980a20d0 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7b90ff9700 1 -- 172.21.15.64:0/4149562471 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b980a2360 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.881+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ba0007620 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7b90ff9700 1 -- 172.21.15.64:0/4149562471 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b98095cd0 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 10 client.26036 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5598806a7c70 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7b90ff9700 1 -- 172.21.15.64:0/4149562471 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba0007620 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7b90ff9700 1 -- 172.21.15.64:0/4149562471 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b98098fc0 con 0x5598808d23e0 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 10 client.26036 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 20 client.26036 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880960cf0 0x5598809630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 4 client.26036 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:09:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55988081ae60 con 0x559880960cf0 2024-05-08T22:09:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.882+0000 7f7bad446700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880960cf0 0x5598809630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.883+0000 7f7bad446700 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880960cf0 0x5598809630e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6018ea6b872a7064 server_cookie=83b7633b6acc32b4 in_seq=0 out_seq=0 2024-05-08T22:09:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.883+0000 7f7b90ff9700 10 client.26036 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7b90ff9700 1 -- 172.21.15.64:0/4149562471 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55988081ae60 con 0x559880960cf0 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7b90ff9700 10 client.26036 handle_command_reply: tid=0 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7bbb864b80 1 client.26036 shutdown 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 msgr2=0x5598808f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f1ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b98094e20 tx=0x7f7b980962f0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 shutdown_connections 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.892+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x559880960cf0 0x5598809630e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598808d32d0 0x5598808f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7b90ff9700 0 client.26036 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598808d1a80 0x5598808f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 1 --2- 172.21.15.64:0/4149562471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598808d23e0 0x5598808f1ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 >> 172.21.15.64:0/4149562471 conn(0x5598808301b0 msgr2=0x5598808d5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 shutdown_connections 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 1 -- 172.21.15.64:0/4149562471 wait complete. 2024-05-08T22:09:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7bbb864b80 20 client.26036 trim_cache size 0 max 16384 2024-05-08T22:09:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:09:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:09:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:09:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b940a12f0 msgr2=0x7f7b940a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b940a12f0 0x7f7b940a37b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4005d10 tx=0x7f7ba4009310 comp rx=0 tx=0).stop 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 msgr2=0x7f7ba819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.893+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 0x7f7ba819daa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800e9f0 tx=0x7f7b9800ed00 comp rx=0 tx=0).stop 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 shutdown_connections 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7b940a12f0 0x7f7b940a37b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba8104ab0 0x7f7ba81a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80ff770 0x7f7ba819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 --2- 172.21.15.64:0/810445802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ba81020e0 0x7f7ba819daa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 >> 172.21.15.64:0/810445802 conn(0x7f7ba80fb390 msgr2=0x7f7ba81004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 shutdown_connections 2024-05-08T22:09:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:26.894+0000 7f7baf6aa700 1 -- 172.21.15.64:0/810445802 wait complete. 2024-05-08T22:09:27.142 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:26 smithi186 ceph-mon[46619]: pgmap v704: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 1.87k op/s 2024-05-08T22:09:27.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:26 smithi064 ceph-mon[42500]: pgmap v704: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 1.87k op/s 2024-05-08T22:09:27.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:26 smithi160 ceph-mon[46962]: pgmap v704: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 1.87k op/s 2024-05-08T22:09:30.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:29 smithi160 ceph-mon[46962]: pgmap v705: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2024-05-08T22:09:30.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:29 smithi186 ceph-mon[46619]: pgmap v705: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2024-05-08T22:09:30.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:29 smithi064 ceph-mon[42500]: pgmap v705: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2024-05-08T22:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 880.0 s, 1361.8 tps, lat 72.509 ms stddev 76.118 2024-05-08T22:09:32.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:31 smithi186 ceph-mon[46619]: pgmap v706: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.52k op/s 2024-05-08T22:09:32.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:31 smithi064 ceph-mon[42500]: pgmap v706: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.52k op/s 2024-05-08T22:09:32.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:31 smithi160 ceph-mon[46962]: pgmap v706: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.52k op/s 2024-05-08T22:09:33.147 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:32 smithi186 ceph-mon[46619]: pgmap v707: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-08T22:09:33.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:32 smithi064 ceph-mon[42500]: pgmap v707: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-08T22:09:33.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:32 smithi160 ceph-mon[46962]: pgmap v707: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-08T22:09:36.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:35 smithi160 ceph-mon[46962]: pgmap v708: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 735 op/s 2024-05-08T22:09:36.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:35 smithi186 ceph-mon[46619]: pgmap v708: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 735 op/s 2024-05-08T22:09:36.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:35 smithi064 ceph-mon[42500]: pgmap v708: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 735 op/s 2024-05-08T22:09:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 885.0 s, 1403.7 tps, lat 69.902 ms stddev 83.377 2024-05-08T22:09:38.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:37 smithi160 ceph-mon[46962]: pgmap v709: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 664 op/s 2024-05-08T22:09:38.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:37 smithi186 ceph-mon[46619]: pgmap v709: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 664 op/s 2024-05-08T22:09:38.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:37 smithi064 ceph-mon[42500]: pgmap v709: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 664 op/s 2024-05-08T22:09:40.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:39 smithi160 ceph-mon[46962]: pgmap v710: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-08T22:09:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 -- 172.21.15.64:0/3104319923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d00ff1c0 msgr2=0x7fd7d00ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/3104319923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d00ff1c0 0x7fd7d00ff5e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 -- 172.21.15.64:0/3104319923 shutdown_connections 2024-05-08T22:09:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/3104319923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d00ff1c0 0x7fd7d00ff5e0 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/3104319923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d01008e0 0x7fd7d0105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/3104319923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d00fff20 0x7fd7d01003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.092+0000 7fd7d85f1700 1 -- 172.21.15.64:0/3104319923 >> 172.21.15.64:0/3104319923 conn(0x7fd7d00faaf0 msgr2=0x7fd7d00fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.108+0000 7fd7d85f1700 1 -- 172.21.15.64:0/3104319923 shutdown_connections 2024-05-08T22:09:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.113+0000 7fd7d85f1700 1 -- 172.21.15.64:0/3104319923 wait complete. 2024-05-08T22:09:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.113+0000 7fd7d85f1700 1 Processor -- start 2024-05-08T22:09:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.119+0000 7fd7d85f1700 1 -- start start 2024-05-08T22:09:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.119+0000 7fd7d85f1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d00fff20 0x7fd7d006faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.119+0000 7fd7d85f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d01008e0 0x7fd7d0070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.119+0000 7fd7d85f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 0x7fd7d01adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.119+0000 7fd7d85f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0070730 con 0x7fd7d01ab700 2024-05-08T22:09:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.119+0000 7fd7d85f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01065a0 con 0x7fd7d01008e0 2024-05-08T22:09:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.120+0000 7fd7d85f1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0106760 con 0x7fd7d00fff20 2024-05-08T22:09:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.122+0000 7fd7d6b8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 0x7fd7d01adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.122+0000 7fd7d6b8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 0x7fd7d01adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36388/0 (socket says 172.21.15.64:36388) 2024-05-08T22:09:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.122+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/174469749 learned_addr learned my addr 172.21.15.64:0/174469749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.123+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/174469749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d00fff20 msgr2=0x7fd7d006faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:09:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.123+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d00fff20 0x7fd7d006faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.123+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/174469749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d01008e0 msgr2=0x7fd7d0070030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:09:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.123+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d01008e0 0x7fd7d0070030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.123+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/174469749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0095f0 con 0x7fd7d01ab700 2024-05-08T22:09:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.123+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 0x7fd7d01adaf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b970 tx=0x7fd7c800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.124+0000 7fd7c77fe700 1 -- 172.21.15.64:0/174469749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800c9b0 con 0x7fd7d01ab700 2024-05-08T22:09:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.124+0000 7fd7c77fe700 1 -- 172.21.15.64:0/174469749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c8014440 con 0x7fd7d01ab700 2024-05-08T22:09:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.124+0000 7fd7c77fe700 1 -- 172.21.15.64:0/174469749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80075d0 con 0x7fd7d01ab700 2024-05-08T22:09:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.124+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0106a40 con 0x7fd7d01ab700 2024-05-08T22:09:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.124+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d0106f00 con 0x7fd7d01ab700 2024-05-08T22:09:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 Processor -- start 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 -- start start 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54ebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ed2d0 0x5598b54ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54430f0 con 0x5598b54ed2d0 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b5440ab0 con 0x5598b54ec3e0 2024-05-08T22:09:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.129+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b543c3a0 con 0x5598b54eba80 2024-05-08T22:09:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.130+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54ebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.130+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54ebea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:41380/0 (socket says 172.21.15.64:41380) 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.130+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3663627388 learned_addr learned my addr 172.21.15.64:0/3663627388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.131+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3663627388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 msgr2=0x5598b54eccf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.131+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54eccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.131+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3663627388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ed2d0 msgr2=0x5598b54ed750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.131+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ed2d0 0x5598b54ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.131+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3663627388 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b52bd350 con 0x5598b54eba80 2024-05-08T22:09:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.132+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54ebea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00017b0 tx=0x7fd7c0001ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4dce2f8902b799 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.135+0000 7fd7c77fe700 1 -- 172.21.15.64:0/174469749 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd7c800cb10 con 0x7fd7d01ab700 2024-05-08T22:09:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.135+0000 7fd7c4ff9700 1 -- 172.21.15.64:0/3663627388 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0004c70 con 0x5598b54eba80 2024-05-08T22:09:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.135+0000 7fd7c4ff9700 1 -- 172.21.15.64:0/3663627388 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c0004dd0 con 0x5598b54eba80 2024-05-08T22:09:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.135+0000 7fd7c4ff9700 1 -- 172.21.15.64:0/3663627388 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0009280 con 0x5598b54eba80 2024-05-08T22:09:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7c77fe700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd7bc0a1320 0x7fd7bc0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3663627388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 msgr2=0x5598b54ebea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54ebea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00017b0 tx=0x7fd7c0001ac0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7c77fe700 1 -- 172.21.15.64:0/174469749 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c8006e40 con 0x7fd7d01ab700 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3663627388 shutdown_connections 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ed2d0 0x5598b54ed750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54ebea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3663627388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54eccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3663627388 >> 172.21.15.64:0/3663627388 conn(0x5598b544a1b0 msgr2=0x5598b54e8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3663627388 shutdown_connections 2024-05-08T22:09:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.136+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3663627388 wait complete. 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 -- start start 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 Processor -- start 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ed2d0 0x5598b54fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b53d8500 con 0x5598b54eba80 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b53d81a0 con 0x5598b54ec3e0 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b52bf640 con 0x5598b54ed2d0 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.137+0000 7fd7e47abb80 10 client.0 _ll_register_callbacks cb 0x5598b53e1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d638d700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd7bc0a1320 0x7fd7bc0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:37302/0 (socket says 172.21.15.64:37302) 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/1289841964 learned_addr learned my addr 172.21.15.64:0/1289841964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ed2d0 0x5598b54fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/1289841964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ed2d0 msgr2=0x5598b54fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ed2d0 0x5598b54fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/1289841964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 msgr2=0x5598b54f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.138+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/1289841964 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c0001490 con 0x5598b54ec3e0 2024-05-08T22:09:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54f5f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0004da0 tx=0x7fd7c0007300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7091b8d31bb66a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T22:09:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7d638d700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd7bc0a1320 0x7fd7bc0a37e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006010 tx=0x7fd7cc005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/1289841964 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0019040 con 0x5598b54ec3e0 2024-05-08T22:09:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/1289841964 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c0009000 con 0x5598b54ec3e0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598b5395210 con 0x5598b54ec3e0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/1289841964 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c0008e20 con 0x5598b54ec3e0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7e47abb80 10 client.26042 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.139+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5598b5269b00 con 0x5598b54ec3e0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/1289841964 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598b5395210 con 0x5598b54ec3e0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/1289841964 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7c00158e0 con 0x5598b54ec3e0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7e47abb80 10 client.26042 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7e47abb80 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7e47abb80 20 client.26042 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b55016f0 0x5598b5503b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7e47abb80 4 client.26042 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.140+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598b54bb7b0 con 0x5598b55016f0 2024-05-08T22:09:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.141+0000 7fd7d638d700 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b55016f0 0x5598b5503b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.147 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:39 smithi186 ceph-mon[46619]: pgmap v710: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.141+0000 7fd7b9ffb700 10 client.26042 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.141+0000 7fd7d638d700 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b55016f0 0x5598b5503b10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95d8d487c5716c90 server_cookie=1a3528ffc7593689 in_seq=0 out_seq=0 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/1289841964 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598b54bb7b0 con 0x5598b55016f0 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7b9ffb700 10 client.26042 handle_command_reply: tid=0 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 client.26042 shutdown 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 msgr2=0x5598b54f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54f5f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0004da0 tx=0x7fd7c0007300 comp rx=0 tx=0).stop 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 shutdown_connections 2024-05-08T22:09:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b55016f0 0x5598b5503b10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ed2d0 0x5598b54fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/1289841964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ec3e0 0x5598b54f5f10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 >> 172.21.15.64:0/1289841964 conn(0x5598b544a1b0 msgr2=0x5598b54e6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.142+0000 7fd7b9ffb700 0 client.26042 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.143+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 shutdown_connections 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.143+0000 7fd7e47abb80 1 -- 172.21.15.64:0/1289841964 wait complete. 2024-05-08T22:09:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.143+0000 7fd7e47abb80 20 client.26042 trim_cache size 0 max 16384 2024-05-08T22:09:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 Processor -- start 2024-05-08T22:09:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 -- start start 2024-05-08T22:09:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ec3e0 0x5598b54fce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b5508470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54ec7c0 con 0x5598b54eba80 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54f5540 con 0x5598b54ed2d0 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.164+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54e6690 con 0x5598b54ec3e0 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b5508470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b5508470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:37306/0 (socket says 172.21.15.64:37306) 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ec3e0 0x5598b54fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/3172339952 learned_addr learned my addr 172.21.15.64:0/3172339952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d638d700 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/3172339952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ec3e0 msgr2=0x5598b54fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ec3e0 0x5598b54fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/3172339952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 msgr2=0x5598b54ed6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54ed6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 -- 172.21.15.64:0/3172339952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b5402500 con 0x5598b54ed2d0 2024-05-08T22:09:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ec3e0 0x5598b54fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b5508470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800be50 tx=0x7fd7c8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7abffd03ba2bf6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.165+0000 7fd7c57fa700 1 -- 172.21.15.64:0/3172339952 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80a15f0 con 0x5598b54ed2d0 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7c57fa700 1 -- 172.21.15.64:0/3172339952 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c8097510 con 0x5598b54ed2d0 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7c57fa700 1 -- 172.21.15.64:0/3172339952 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80a13c0 con 0x5598b54ed2d0 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3172339952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 msgr2=0x5598b5508470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b5508470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800be50 tx=0x7fd7c8004ad0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3172339952 shutdown_connections 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b5508470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54eba80 0x5598b54ed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.166+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3172339952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54ec3e0 0x5598b54fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3172339952 >> 172.21.15.64:0/3172339952 conn(0x5598b544a1b0 msgr2=0x5598b54f3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3172339952 shutdown_connections 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3172339952 wait complete. 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- start start 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 Processor -- start 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 0x5598b54fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b54fc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54fc970 con 0x5598b54ec3e0 2024-05-08T22:09:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54fcae0 con 0x5598b54ed2d0 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b54fcca0 con 0x5598b54eba80 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.167+0000 7fd7e47abb80 10 client.0 _ll_register_callbacks cb 0x5598b53e1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 0x5598b54fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 0x5598b54fb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36424/0 (socket says 172.21.15.64:36424) 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3265710497 learned_addr learned my addr 172.21.15.64:0/3265710497 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d6b8e700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b54fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d638d700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3265710497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 msgr2=0x5598b54fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 msgr2=0x5598b54fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b54fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.168+0000 7fd7d5b8c700 1 -- 172.21.15.64:0/3265710497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598b5269b00 con 0x5598b54ec3e0 2024-05-08T22:09:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.169+0000 7fd7d5b8c700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 0x5598b54fb5e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009f40 tx=0x7fd7c000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2b2cd33c6c94c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.169+0000 7fd7b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.170+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/3265710497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0019040 con 0x5598b54ec3e0 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.170+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/3265710497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c000e8b0 con 0x5598b54ec3e0 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.170+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7c0004dd0 con 0x5598b54ec3e0 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.170+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/3265710497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c0037030 con 0x5598b54ec3e0 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.170+0000 7fd7e47abb80 10 client.16254 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.170+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fd7d0106f00 con 0x5598b54ec3e0 2024-05-08T22:09:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/3265710497 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c0004dd0 con 0x5598b54ec3e0 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/3265710497 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7c002f020 con 0x5598b54ec3e0 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7e47abb80 10 client.16254 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7e47abb80 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7e47abb80 20 client.16254 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b5558b70 0x5598b555af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7e47abb80 4 client.16254 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.171+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598b5434e60 con 0x5598b5558b70 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.172+0000 7fd7d638d700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b5558b70 0x5598b555af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.172+0000 7fd7b9ffb700 10 client.16254 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.172+0000 7fd7d638d700 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b5558b70 0x5598b555af60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fa1cc1df06f5bf server_cookie=1c511915af48a61 in_seq=0 out_seq=0 2024-05-08T22:09:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7b9ffb700 1 -- 172.21.15.64:0/3265710497 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598b5434e60 con 0x5598b5558b70 2024-05-08T22:09:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7b9ffb700 10 client.16254 handle_command_reply: tid=0 2024-05-08T22:09:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 client.16254 shutdown 2024-05-08T22:09:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 msgr2=0x5598b54fb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 0x5598b54fb5e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009f40 tx=0x7fd7c000e6c0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 shutdown_connections 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5598b54ed2d0 0x5598b54fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x5598b5558b70 0x5598b555af60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598b54eba80 0x5598b54fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 --2- 172.21.15.64:0/3265710497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598b54ec3e0 0x5598b54fb5e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 >> 172.21.15.64:0/3265710497 conn(0x5598b544a1b0 msgr2=0x5598b5508470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.192+0000 7fd7b9ffb700 0 client.16254 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.193+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 shutdown_connections 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.193+0000 7fd7e47abb80 1 -- 172.21.15.64:0/3265710497 wait complete. 2024-05-08T22:09:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.193+0000 7fd7e47abb80 20 client.16254 trim_cache size 0 max 16384 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.194+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd7bc0a1320 msgr2=0x7fd7bc0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.194+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd7bc0a1320 0x7fd7bc0a37e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006010 tx=0x7fd7cc005c00 comp rx=0 tx=0).stop 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.194+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 msgr2=0x7fd7d01adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.194+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 0x7fd7d01adaf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b970 tx=0x7fd7c800bd30 comp rx=0 tx=0).stop 2024-05-08T22:09:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 shutdown_connections 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd7bc0a1320 0x7fd7bc0a37e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7d01ab700 0x7fd7d01adaf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d00fff20 0x7fd7d006faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 --2- 172.21.15.64:0/174469749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7d01008e0 0x7fd7d0070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 >> 172.21.15.64:0/174469749 conn(0x7fd7d00faaf0 msgr2=0x7fd7d0101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 shutdown_connections 2024-05-08T22:09:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:40.198+0000 7fd7d85f1700 1 -- 172.21.15.64:0/174469749 wait complete. 2024-05-08T22:09:40.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:39 smithi064 ceph-mon[42500]: pgmap v710: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-08T22:09:41.484 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 890.0 s, 1589.4 tps, lat 64.632 ms stddev 77.486 2024-05-08T22:09:42.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:41 smithi160 ceph-mon[46962]: pgmap v711: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 688 op/s 2024-05-08T22:09:42.145 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:41 smithi186 ceph-mon[46619]: pgmap v711: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 688 op/s 2024-05-08T22:09:42.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:41 smithi064 ceph-mon[42500]: pgmap v711: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 688 op/s 2024-05-08T22:09:43.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:09:43.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:42 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:09:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:42 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:09:43.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:43.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:42 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:43 smithi160 ceph-mon[46962]: pgmap v712: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 MiB/s wr, 673 op/s 2024-05-08T22:09:44.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:09:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:09:44.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:43 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:43 smithi186 ceph-mon[46619]: pgmap v712: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 MiB/s wr, 673 op/s 2024-05-08T22:09:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:09:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:09:44.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:43 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:43 smithi064 ceph-mon[42500]: pgmap v712: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 MiB/s wr, 673 op/s 2024-05-08T22:09:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:09:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:09:44.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:43 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' 2024-05-08T22:09:46.145 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:45 smithi186 ceph-mon[46619]: pgmap v713: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 700 op/s 2024-05-08T22:09:46.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:45 smithi064 ceph-mon[42500]: pgmap v713: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 700 op/s 2024-05-08T22:09:46.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:45 smithi160 ceph-mon[46962]: pgmap v713: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 700 op/s 2024-05-08T22:09:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 895.0 s, 1563.5 tps, lat 64.192 ms stddev 72.092 2024-05-08T22:09:47.146 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:46 smithi186 ceph-mon[46619]: pgmap v714: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 738 op/s 2024-05-08T22:09:47.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:46 smithi064 ceph-mon[42500]: pgmap v714: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 738 op/s 2024-05-08T22:09:47.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:46 smithi160 ceph-mon[46962]: pgmap v714: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 738 op/s 2024-05-08T22:09:50.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:49 smithi160 ceph-mon[46962]: pgmap v715: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 MiB/s wr, 691 op/s 2024-05-08T22:09:50.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:49 smithi186 ceph-mon[46619]: pgmap v715: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 MiB/s wr, 691 op/s 2024-05-08T22:09:50.224 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:49 smithi064 ceph-mon[42500]: pgmap v715: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 MiB/s wr, 691 op/s 2024-05-08T22:09:51.475 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 900.0 s, 1303.0 tps, lat 75.932 ms stddev 87.941 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:transaction type: 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:scaling factor: 32 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:query mode: simple 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:number of clients: 100 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:number of threads: 4 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:duration: 900 s 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:number of transactions actually processed: 1938244 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:latency average = 46.435 ms 2024-05-08T22:09:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:latency stddev = 65.666 ms 2024-05-08T22:09:51.747 INFO:teuthology.orchestra.run.smithi064.stdout:tps = 2152.958458 (including connections establishing) 2024-05-08T22:09:51.747 INFO:teuthology.orchestra.run.smithi064.stdout:tps = 2152.964198 (excluding connections establishing) 2024-05-08T22:09:51.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-08T22:09:52.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:51 smithi186 ceph-mon[46619]: pgmap v716: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 701 op/s 2024-05-08T22:09:52.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:51 smithi064 ceph-mon[42500]: pgmap v716: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 701 op/s 2024-05-08T22:09:52.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:51 smithi160 ceph-mon[46962]: pgmap v716: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 701 op/s 2024-05-08T22:09:53.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:52 smithi064 ceph-mon[42500]: pgmap v717: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 668 op/s 2024-05-08T22:09:53.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:52 smithi160 ceph-mon[46962]: pgmap v717: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 668 op/s 2024-05-08T22:09:53.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:52 smithi186 ceph-mon[46619]: pgmap v717: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 668 op/s 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 -- 172.21.15.64:0/302983333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf00786a0 msgr2=0x7f1cf007ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/302983333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf00786a0 0x7f1cf007ab30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0007570 tx=0x7f1ce0007880 comp rx=0 tx=0).stop 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 -- 172.21.15.64:0/302983333 shutdown_connections 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/302983333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf00786a0 0x7f1cf007ab30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/302983333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0072da0 0x7f1cf00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/302983333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf0075cd0 0x7f1cf0078160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 -- 172.21.15.64:0/302983333 >> 172.21.15.64:0/302983333 conn(0x7f1cf006df30 msgr2=0x7f1cf0070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 -- 172.21.15.64:0/302983333 shutdown_connections 2024-05-08T22:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.446+0000 7f1cf83c0700 1 -- 172.21.15.64:0/302983333 wait complete. 2024-05-08T22:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 Processor -- start 2024-05-08T22:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 -- start start 2024-05-08T22:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf0072da0 0x7f1cf012bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0075cd0 0x7f1cf012e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00786a0 0x7f1cf0130a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0131220 con 0x7f1cf00786a0 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0131390 con 0x7f1cf0072da0 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.447+0000 7f1cf83c0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0131550 con 0x7f1cf0075cd0 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.450+0000 7f1cf695d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00786a0 0x7f1cf0130a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.450+0000 7f1cf595b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0075cd0 0x7f1cf012e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.450+0000 7f1cf595b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0075cd0 0x7f1cf012e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42158/0 (socket says 172.21.15.64:42158) 2024-05-08T22:09:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.450+0000 7f1cf595b700 1 -- 172.21.15.64:0/2519349297 learned_addr learned my addr 172.21.15.64:0/2519349297 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.466+0000 7f1cf615c700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf0072da0 0x7f1cf012bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.466+0000 7f1cf695d700 1 -- 172.21.15.64:0/2519349297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0075cd0 msgr2=0x7f1cf012e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.466+0000 7f1cf695d700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0075cd0 0x7f1cf012e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.466+0000 7f1cf695d700 1 -- 172.21.15.64:0/2519349297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf0072da0 msgr2=0x7f1cf012bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.466+0000 7f1cf695d700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf0072da0 0x7f1cf012bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.466+0000 7f1cf695d700 1 -- 172.21.15.64:0/2519349297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce0007220 con 0x7f1cf00786a0 2024-05-08T22:09:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.479+0000 7f1cf695d700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00786a0 0x7f1cf0130a80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000a270 tx=0x7f1ce000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.493+0000 7f1ce77fe700 1 -- 172.21.15.64:0/2519349297 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000f040 con 0x7f1cf00786a0 2024-05-08T22:09:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.493+0000 7f1ce77fe700 1 -- 172.21.15.64:0/2519349297 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce000ce60 con 0x7f1cf00786a0 2024-05-08T22:09:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.493+0000 7f1ce77fe700 1 -- 172.21.15.64:0/2519349297 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00038e0 con 0x7f1cf00786a0 2024-05-08T22:09:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.493+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf01317d0 con 0x7f1cf00786a0 2024-05-08T22:09:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.493+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf0131c90 con 0x7f1cf00786a0 2024-05-08T22:09:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.499+0000 7f1d0457ab80 1 Processor -- start 2024-05-08T22:09:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.499+0000 7f1d0457ab80 1 -- start start 2024-05-08T22:09:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.499+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.499+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.500+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93db750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.500+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93310f0 con 0x55ecb93d9a80 2024-05-08T22:09:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.500+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb932eab0 con 0x55ecb93db2d0 2024-05-08T22:09:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.500+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb932a3a0 con 0x55ecb93da3e0 2024-05-08T22:09:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.503+0000 7f1cf615c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.503+0000 7f1cf615c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93dacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.64:42174/0 (socket says 172.21.15.64:42174) 2024-05-08T22:09:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.503+0000 7f1cf615c700 1 -- 172.21.15.64:0/2966853900 learned_addr learned my addr 172.21.15.64:0/2966853900 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.503+0000 7f1cf595b700 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.508+0000 7f1cf615c700 1 -- 172.21.15.64:0/2966853900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 msgr2=0x55ecb93db750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.508+0000 7f1cf615c700 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.508+0000 7f1cf615c700 1 -- 172.21.15.64:0/2966853900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 msgr2=0x55ecb93d9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.508+0000 7f1cf615c700 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93d9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.508+0000 7f1cf615c700 1 -- 172.21.15.64:0/2966853900 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb91ab350 con 0x55ecb93da3e0 2024-05-08T22:09:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.509+0000 7f1ce77fe700 1 -- 172.21.15.64:0/2519349297 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1ce0003a40 con 0x7f1cf00786a0 2024-05-08T22:09:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.510+0000 7f1ce77fe700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1cdc0a1320 0x7f1cdc0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.510+0000 7f1ce77fe700 1 -- 172.21.15.64:0/2519349297 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ce00cc630 con 0x7f1cf00786a0 2024-05-08T22:09:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.510+0000 7f1cf595b700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1cdc0a1320 0x7f1cdc0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.512+0000 7f1cf595b700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1cdc0a1320 0x7f1cdc0a37e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55ecb93daab0 tx=0x7f1ce800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.526+0000 7f1cf615c700 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93dacf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007810 tx=0x7f1cec007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34290a1b1409759b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.538+0000 7f1ce4da4700 1 -- 172.21.15.64:0/2966853900 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0125f0 con 0x55ecb93da3e0 2024-05-08T22:09:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.538+0000 7f1ce4da4700 1 -- 172.21.15.64:0/2966853900 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cec012c10 con 0x55ecb93da3e0 2024-05-08T22:09:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.538+0000 7f1ce4da4700 1 -- 172.21.15.64:0/2966853900 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec013e40 con 0x55ecb93da3e0 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 -- 172.21.15.64:0/2966853900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 msgr2=0x55ecb93dacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93dacf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007810 tx=0x7f1cec007b20 comp rx=0 tx=0).stop 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 -- 172.21.15.64:0/2966853900 shutdown_connections 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93db750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93d9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/2966853900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93dacf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.539+0000 7f1d0457ab80 1 -- 172.21.15.64:0/2966853900 >> 172.21.15.64:0/2966853900 conn(0x55ecb93381b0 msgr2=0x55ecb93d6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.550+0000 7f1d0457ab80 1 -- 172.21.15.64:0/2966853900 shutdown_connections 2024-05-08T22:09:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.550+0000 7f1d0457ab80 1 -- 172.21.15.64:0/2966853900 wait complete. 2024-05-08T22:09:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.550+0000 7f1d0457ab80 1 -- start start 2024-05-08T22:09:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 Processor -- start 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93db2d0 0x55ecb93eae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb92c6500 con 0x55ecb93d9a80 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb92c61a0 con 0x55ecb93da3e0 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb91ad640 con 0x55ecb93db2d0 2024-05-08T22:09:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.551+0000 7f1d0457ab80 10 client.0 _ll_register_callbacks cb 0x55ecb92cfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.554+0000 7f1cf615c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.554+0000 7f1cf615c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48678/0 (socket says 172.21.15.64:48678) 2024-05-08T22:09:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.554+0000 7f1cf615c700 1 -- 172.21.15.64:0/1510609468 learned_addr learned my addr 172.21.15.64:0/1510609468 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.557+0000 7f1cf695d700 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93db2d0 0x55ecb93eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.560+0000 7f1cf615c700 1 -- 172.21.15.64:0/1510609468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93db2d0 msgr2=0x55ecb93eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.560+0000 7f1cf615c700 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93db2d0 0x55ecb93eae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.560+0000 7f1cf615c700 1 -- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 msgr2=0x55ecb93e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.560+0000 7f1cf615c700 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.560+0000 7f1cf615c700 1 -- 172.21.15.64:0/1510609468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec00e040 con 0x55ecb93d9a80 2024-05-08T22:09:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.564+0000 7f1cf615c700 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e38f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007070 tx=0x7f1cec0136f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abf83279b7a7330 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.569+0000 7f1cd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:09:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.569+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/1510609468 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec012ec0 con 0x55ecb93d9a80 2024-05-08T22:09:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.569+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/1510609468 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cec013d50 con 0x55ecb93d9a80 2024-05-08T22:09:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.572+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecb9283210 con 0x55ecb93d9a80 2024-05-08T22:09:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.583+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/1510609468 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cec0125f0 con 0x55ecb93d9a80 2024-05-08T22:09:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.583+0000 7f1d0457ab80 10 client.16266 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.583+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ecb9157b00 con 0x55ecb93d9a80 2024-05-08T22:09:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.592+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/1510609468 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecb9283210 con 0x55ecb93d9a80 2024-05-08T22:09:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.592+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/1510609468 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1cec031c20 con 0x55ecb93d9a80 2024-05-08T22:09:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.593+0000 7f1d0457ab80 10 client.16266 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.593+0000 7f1d0457ab80 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.593+0000 7f1d0457ab80 20 client.16266 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.593+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb93ef670 0x55ecb93f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.593+0000 7f1d0457ab80 4 client.16266 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.593+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecb93a97b0 con 0x55ecb93ef670 2024-05-08T22:09:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.605+0000 7f1cf595b700 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb93ef670 0x55ecb93f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.626+0000 7f1cf595b700 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb93ef670 0x55ecb93f1a90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7d8b6d78a40a2a server_cookie=48327da334e40ac5 in_seq=0 out_seq=0 2024-05-08T22:09:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1cd9ffb700 10 client.16266 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/1510609468 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecb93a97b0 con 0x55ecb93ef670 2024-05-08T22:09:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1cd9ffb700 10 client.16266 handle_command_reply: tid=0 2024-05-08T22:09:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 client.16266 shutdown 2024-05-08T22:09:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 msgr2=0x55ecb93e38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e38f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007070 tx=0x7f1cec0136f0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 shutdown_connections 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb93ef670 0x55ecb93f1a90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93db2d0 0x55ecb93eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e38f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/1510609468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 >> 172.21.15.64:0/1510609468 conn(0x55ecb93381b0 msgr2=0x55ecb93d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.634+0000 7f1cd9ffb700 0 client.16266 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.635+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 shutdown_connections 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.635+0000 7f1d0457ab80 1 -- 172.21.15.64:0/1510609468 wait complete. 2024-05-08T22:09:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.635+0000 7f1d0457ab80 20 client.16266 trim_cache size 0 max 16384 2024-05-08T22:09:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 Processor -- start 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 -- start start 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93d9a80 0x55ecb93eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93db2d0 0x55ecb93eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93e2390 con 0x55ecb93db2d0 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93e14f0 con 0x55ecb93da3e0 2024-05-08T22:09:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93dee00 con 0x55ecb93d9a80 2024-05-08T22:09:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1cf595b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1cf595b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93db6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:49516/0 (socket says 172.21.15.64:49516) 2024-05-08T22:09:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.664+0000 7f1cf595b700 1 -- 172.21.15.64:0/3199257755 learned_addr learned my addr 172.21.15.64:0/3199257755 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.675+0000 7f1cf615c700 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93d9a80 0x55ecb93eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.675+0000 7f1cf695d700 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93db2d0 0x55ecb93eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.686+0000 7f1cf595b700 1 -- 172.21.15.64:0/3199257755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93d9a80 msgr2=0x55ecb93eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.686+0000 7f1cf595b700 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93d9a80 0x55ecb93eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.686+0000 7f1cf595b700 1 -- 172.21.15.64:0/3199257755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93db2d0 msgr2=0x55ecb93eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.686+0000 7f1cf595b700 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93db2d0 0x55ecb93eab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.686+0000 7f1cf595b700 1 -- 172.21.15.64:0/3199257755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb92f0500 con 0x55ecb93da3e0 2024-05-08T22:09:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.699+0000 7f1cf595b700 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93db6b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800b010 tx=0x7f1ce8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d0e21a1017d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.708+0000 7f1ce55a5700 1 -- 172.21.15.64:0/3199257755 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8017d20 con 0x55ecb93da3e0 2024-05-08T22:09:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.708+0000 7f1ce55a5700 1 -- 172.21.15.64:0/3199257755 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce802b070 con 0x55ecb93da3e0 2024-05-08T22:09:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.708+0000 7f1ce55a5700 1 -- 172.21.15.64:0/3199257755 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8005020 con 0x55ecb93da3e0 2024-05-08T22:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.709+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3199257755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 msgr2=0x55ecb93db6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.709+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93db6b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800b010 tx=0x7f1ce8004ad0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.712+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3199257755 shutdown_connections 2024-05-08T22:09:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.712+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93db2d0 0x55ecb93eab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.712+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93d9a80 0x55ecb93eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.712+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3199257755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93da3e0 0x55ecb93db6b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.712+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3199257755 >> 172.21.15.64:0/3199257755 conn(0x55ecb93381b0 msgr2=0x55ecb93e2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.718+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3199257755 shutdown_connections 2024-05-08T22:09:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3199257755 wait complete. 2024-05-08T22:09:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 -- start start 2024-05-08T22:09:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 Processor -- start 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93ea020 con 0x55ecb93d9a80 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93ea190 con 0x55ecb93db2d0 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb93ea350 con 0x55ecb93da3e0 2024-05-08T22:09:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.729+0000 7f1d0457ab80 10 client.0 _ll_register_callbacks cb 0x55ecb92cfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.735+0000 7f1cf615c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.735+0000 7f1cf615c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48686/0 (socket says 172.21.15.64:48686) 2024-05-08T22:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.735+0000 7f1cf615c700 1 -- 172.21.15.64:0/3092537809 learned_addr learned my addr 172.21.15.64:0/3092537809 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.741+0000 7f1cf695d700 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.741+0000 7f1cf615c700 1 -- 172.21.15.64:0/3092537809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 msgr2=0x55ecb93e8b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.741+0000 7f1cf615c700 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93e8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.741+0000 7f1cf615c700 1 -- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 msgr2=0x55ecb93e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.741+0000 7f1cf615c700 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.741+0000 7f1cf615c700 1 -- 172.21.15.64:0/3092537809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce8011040 con 0x55ecb93d9a80 2024-05-08T22:09:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.771+0000 7f1cf615c700 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e85c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1cec013870 tx=0x7f1cec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5f21a42bb95405 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.771+0000 7f1cd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-08T22:09:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.772+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/3092537809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec027a90 con 0x55ecb93d9a80 2024-05-08T22:09:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.772+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/3092537809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cec000e40 con 0x55ecb93d9a80 2024-05-08T22:09:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.772+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cec012c10 con 0x55ecb93d9a80 2024-05-08T22:09:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.773+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/3092537809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cec012ec0 con 0x55ecb93d9a80 2024-05-08T22:09:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.773+0000 7f1d0457ab80 10 client.16272 fetch_fsmap learned FSMap version 47 2024-05-08T22:09:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.773+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ecb91afc70 con 0x55ecb93d9a80 2024-05-08T22:09:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/3092537809 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cec012c10 con 0x55ecb93d9a80 2024-05-08T22:09:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/3092537809 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1cec01cbc0 con 0x55ecb93d9a80 2024-05-08T22:09:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1d0457ab80 10 client.16272 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:09:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1d0457ab80 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1d0457ab80 20 client.16272 populate_metadata read hostname 'smithi064' 2024-05-08T22:09:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb9446030 0x55ecb9448450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1d0457ab80 4 client.16272 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:09:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.778+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 --> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecb9322e60 con 0x55ecb9446030 2024-05-08T22:09:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.779+0000 7f1cf595b700 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb9446030 0x55ecb9448450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.817+0000 7f1cd9ffb700 10 client.16272 ms_handle_connect on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:53.817+0000 7f1cf595b700 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb9446030 0x55ecb9448450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b189d7285af4ae82 server_cookie=6b7d4c683055d6c in_seq=0 out_seq=0 2024-05-08T22:09:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.704+0000 7f1cd9ffb700 1 -- 172.21.15.64:0/3092537809 <== mds.0 v2:172.21.15.160:6838/2466892222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecb9322e60 con 0x55ecb9446030 2024-05-08T22:09:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.704+0000 7f1cd9ffb700 10 client.16272 handle_command_reply: tid=0 2024-05-08T22:09:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.704+0000 7f1d0457ab80 1 client.16272 shutdown 2024-05-08T22:09:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.726+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 msgr2=0x55ecb93e85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.726+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e85c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1cec013870 tx=0x7f1cec000bc0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 shutdown_connections 2024-05-08T22:09:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecb93db2d0 0x55ecb93e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.160:6838/2466892222,v1:172.21.15.160:6839/2466892222] conn(0x55ecb9446030 0x55ecb9448450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ecb93d9a80 0x55ecb93e85c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1d0457ab80 1 --2- 172.21.15.64:0/3092537809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecb93da3e0 0x55ecb93e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 >> 172.21.15.64:0/3092537809 conn(0x55ecb93381b0 msgr2=0x55ecb93f1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.736+0000 7f1cd9ffb700 0 client.16272 ms_handle_reset on v2:172.21.15.160:6838/2466892222 2024-05-08T22:09:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.758+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 shutdown_connections 2024-05-08T22:09:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.758+0000 7f1d0457ab80 1 -- 172.21.15.64:0/3092537809 wait complete. 2024-05-08T22:09:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.758+0000 7f1d0457ab80 20 client.16272 trim_cache size 0 max 16384 2024-05-08T22:09:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-08T22:09:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-08T22:09:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-08T22:09:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-08T22:09:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.762+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1cdc0a1320 msgr2=0x7f1cdc0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.762+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1cdc0a1320 0x7f1cdc0a37e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55ecb93daab0 tx=0x7f1ce800b040 comp rx=0 tx=0).stop 2024-05-08T22:09:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.762+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00786a0 msgr2=0x7f1cf0130a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.762+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00786a0 0x7f1cf0130a80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000a270 tx=0x7f1ce000c830 comp rx=0 tx=0).stop 2024-05-08T22:09:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.774+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 shutdown_connections 2024-05-08T22:09:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.774+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f1cdc0a1320 0x7f1cdc0a37e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.774+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf0072da0 0x7f1cf012bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.774+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00786a0 0x7f1cf0130a80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.774+0000 7f1cf83c0700 1 --2- 172.21.15.64:0/2519349297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf0075cd0 0x7f1cf012e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.774+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 >> 172.21.15.64:0/2519349297 conn(0x7f1cf006df30 msgr2=0x7f1cf00772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.778+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 shutdown_connections 2024-05-08T22:09:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-08T22:09:54.778+0000 7f1cf83c0700 1 -- 172.21.15.64:0/2519349297 wait complete. 2024-05-08T22:09:56.146 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:55 smithi186 ceph-mon[46619]: pgmap v718: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 698 op/s 2024-05-08T22:09:56.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:55 smithi064 ceph-mon[42500]: pgmap v718: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 698 op/s 2024-05-08T22:09:56.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:55 smithi160 ceph-mon[46962]: pgmap v718: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 698 op/s 2024-05-08T22:09:57.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:56 smithi064 ceph-mon[42500]: pgmap v719: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-08T22:09:57.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:56 smithi160 ceph-mon[46962]: pgmap v719: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-08T22:09:57.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:56 smithi186 ceph-mon[46619]: pgmap v719: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-08T22:10:00.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:09:59 smithi160 ceph-mon[46962]: pgmap v720: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 974 op/s 2024-05-08T22:10:00.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:09:59 smithi186 ceph-mon[46619]: pgmap v720: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 974 op/s 2024-05-08T22:10:00.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:09:59 smithi064 ceph-mon[42500]: pgmap v720: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 974 op/s 2024-05-08T22:10:00.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-08T22:10:00.568 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/: 2024-05-08T22:10:00.568 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.568 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T22:10:00.568 INFO:teuthology.orchestra.run.smithi064.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 .. 2024-05-08T22:10:00.568 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 data 2024-05-08T22:10:00.568 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data: 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:total 31 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 . 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 .. 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 8 21:54 base 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 8 21:54 current_logfiles 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 May 8 21:54 global 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 log 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_commit_ts 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_dynshmem 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 May 8 21:54 pg_hba.conf 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 8 21:54 pg_ident.conf 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 22:10 pg_logical 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 pg_multixact 2024-05-08T22:10:00.569 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 pg_notify 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_replslot 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_serial 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_snapshots 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 22:10 pg_stat 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 22:10 pg_stat_tmp 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 22:10 pg_subtrans 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_tblspc 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pg_twophase 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 8 22:10 pg_wal 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 22:01 pg_xact 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 8 21:54 postgresql.auto.conf 2024-05-08T22:10:00.570 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 May 8 21:54 postgresql.conf 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 8 21:54 postmaster.opts 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base: 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 8 21:54 . 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 1 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 12346 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 May 8 21:54 12347 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 pgsql_tmp 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-08T22:10:00.571 INFO:teuthology.orchestra.run.smithi064.stdout:total 7217 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 . 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 8 21:54 .. 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 112 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 113 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 8 21:54 12182 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12182_fsm 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12182_vm 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12184 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12186 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12187_fsm 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187_vm 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12189 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12191 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192 2024-05-08T22:10:00.572 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12192_fsm 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192_vm 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12194 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12196 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12197_fsm 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197_vm 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12199 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12201 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12202_fsm 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202_vm 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12204 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12206 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207 2024-05-08T22:10:00.573 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12207_fsm 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207_vm 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12209 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12211 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12212 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12214 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12216 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 8 21:54 1247 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1247_fsm 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1247_vm 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 8 21:54 1249 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1249_fsm 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1249_vm 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 8 21:54 1255 2024-05-08T22:10:00.574 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1255_fsm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1255_vm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 21:54 1259 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1259_fsm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1259_vm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417_vm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418_vm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 174 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 175 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2187 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224_vm 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328 2024-05-08T22:10:00.575 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328_vm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336_vm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2337 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2600 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2600_fsm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2600_vm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2601_fsm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601_vm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2602 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2602_fsm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2602_vm 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2603 2024-05-08T22:10:00.576 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2603_fsm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2603_vm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604_vm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2605 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2605_fsm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2605_vm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2606_fsm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606_vm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607_fsm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2607_vm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 8 21:54 2608 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2608_fsm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2608_vm 2024-05-08T22:10:00.577 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 8 21:54 2609 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2609_fsm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2609_vm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610_fsm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2610_vm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611_vm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2612_fsm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612_vm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613_vm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2615_fsm 2024-05-08T22:10:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615_vm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616_fsm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2616_vm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 8 21:54 2617 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2617_fsm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2617_vm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 8 21:54 2618 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2618_fsm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2618_vm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 8 21:54 2619 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2619_fsm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2619_vm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620_vm 2024-05-08T22:10:00.579 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2650 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2651 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2652 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2653 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2654 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2655 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2656 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2657 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 8 21:54 2658 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2659 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2660 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2661 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2662 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2663 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2664 2024-05-08T22:10:00.580 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2665 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2666 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2667 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2668 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2669 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2670 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 8 21:54 2673 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 May 8 21:54 2674 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 8 21:54 2675 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2678 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2679 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2680 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2681 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2682 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2683 2024-05-08T22:10:00.581 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2684 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2685 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2686 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2687 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2688 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2689 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2690 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 8 21:54 2691 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2692 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2693 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2696 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2699 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2701 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2702 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2703 2024-05-08T22:10:00.582 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2704 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2753 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2753_fsm 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2753_vm 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2754 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2755 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2756 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2757 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830_vm 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2831 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832_vm 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2833 2024-05-08T22:10:00.583 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834_vm 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2835 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836_vm 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2837 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 8 21:54 2838 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2838_fsm 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2838_vm 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2839 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840_fsm 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2840_vm 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2841 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995 2024-05-08T22:10:00.584 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995_vm 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2996 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3079_fsm 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079_vm 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3080 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3081 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3085 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118_vm 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3119 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3164 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256_vm 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3257 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3258 2024-05-08T22:10:00.585 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350_vm 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3351 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3379 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3380 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381_vm 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3394 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3394_fsm 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3394_vm 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3395 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439_vm 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3440 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 3455 2024-05-08T22:10:00.586 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3456 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3456_fsm 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3456_vm 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466_vm 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3467 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3468 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501_vm 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3502 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3503 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3534 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3541_fsm 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541_vm 2024-05-08T22:10:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3542 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3574 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3575 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576_vm 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596_vm 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3597 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598_vm 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3599 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3600_fsm 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600_vm 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601 2024-05-08T22:10:00.588 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3601_fsm 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601_vm 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3602_fsm 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602_vm 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3603 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3603_fsm 2024-05-08T22:10:00.589 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3603_vm 2024-05-08T22:10:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3604 2024-05-08T22:10:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3605 2024-05-08T22:10:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3606 2024-05-08T22:10:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3607 2024-05-08T22:10:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3608 2024-05-08T22:10:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 3609 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3712 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3764_fsm 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764_vm 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3766 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3767 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3997 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 5002 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 548 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 549 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102_vm 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104_vm 2024-05-08T22:10:00.591 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106_vm 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6110 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6111 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6112 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6113 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6117 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826_vm 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 827 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 828 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/12346: 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:total 7217 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 May 8 21:54 . 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 8 21:54 .. 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 112 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 113 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 8 21:54 12182 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12182_fsm 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12182_vm 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12184 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12186 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12187_fsm 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187_vm 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12189 2024-05-08T22:10:00.593 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12191 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12192_fsm 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192_vm 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12194 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12196 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12197_fsm 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197_vm 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12199 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12201 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12202_fsm 2024-05-08T22:10:00.594 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202_vm 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12204 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12206 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12207_fsm 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207_vm 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12209 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12211 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12212 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12214 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12216 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 8 21:54 1247 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1247_fsm 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1247_vm 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 8 21:54 1249 2024-05-08T22:10:00.595 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1249_fsm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1249_vm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 8 21:54 1255 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1255_fsm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1255_vm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 21:54 1259 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1259_fsm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1259_vm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417_vm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418_vm 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 174 2024-05-08T22:10:00.596 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 175 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2187 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224_vm 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328_vm 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336_vm 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2337 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2600 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2600_fsm 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2600_vm 2024-05-08T22:10:00.597 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2601_fsm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601_vm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2602 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2602_fsm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2602_vm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2603 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2603_fsm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2603_vm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604_vm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2605 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2605_fsm 2024-05-08T22:10:00.598 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2605_vm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2606_fsm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606_vm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607_fsm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2607_vm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 8 21:54 2608 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2608_fsm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2608_vm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 8 21:54 2609 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2609_fsm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2609_vm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610_fsm 2024-05-08T22:10:00.599 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2610_vm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611_vm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2612_fsm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612_vm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613_vm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2615_fsm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615_vm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616_fsm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2616_vm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 8 21:54 2617 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2617_fsm 2024-05-08T22:10:00.600 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2617_vm 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 8 21:54 2618 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2618_fsm 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2618_vm 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 8 21:54 2619 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2619_fsm 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2619_vm 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620_vm 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2650 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2651 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2652 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2653 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2654 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2655 2024-05-08T22:10:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2656 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2657 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 8 21:54 2658 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2659 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2660 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2661 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2662 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2663 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2664 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2665 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2666 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2667 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2668 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2669 2024-05-08T22:10:00.602 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2670 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 8 21:54 2673 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 May 8 21:54 2674 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 8 21:54 2675 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2678 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2679 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2680 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2681 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2682 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2683 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2684 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2685 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2686 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2687 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2688 2024-05-08T22:10:00.603 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2689 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2690 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 8 21:54 2691 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2692 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2693 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2696 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2699 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2701 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2702 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2703 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2704 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2753 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2753_fsm 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2753_vm 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2754 2024-05-08T22:10:00.604 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2755 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2756 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2757 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830_vm 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2831 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832_vm 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2833 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834_vm 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2835 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836_vm 2024-05-08T22:10:00.605 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2837 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 8 21:54 2838 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2838_fsm 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2838_vm 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2839 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840_fsm 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2840_vm 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2841 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995_vm 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2996 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3079_fsm 2024-05-08T22:10:00.606 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079_vm 2024-05-08T22:10:00.607 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3080 2024-05-08T22:10:00.607 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3081 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3085 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118_vm 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3119 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3164 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256_vm 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3257 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3258 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350_vm 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3351 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3379 2024-05-08T22:10:00.609 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3380 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381_vm 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3394 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3394_fsm 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3394_vm 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3395 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439_vm 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3440 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 3455 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3456 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3456_fsm 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3456_vm 2024-05-08T22:10:00.610 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466_vm 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3467 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3468 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501_vm 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3502 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3503 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3534 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3541_fsm 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541_vm 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3542 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3574 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3575 2024-05-08T22:10:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576_vm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596_vm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3597 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598_vm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3599 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3600_fsm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600_vm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3601_fsm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601_vm 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602 2024-05-08T22:10:00.612 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3602_fsm 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602_vm 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3603 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3603_fsm 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3603_vm 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3604 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3605 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3606 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3607 2024-05-08T22:10:00.613 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3608 2024-05-08T22:10:00.615 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 3609 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3712 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3764_fsm 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764_vm 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3766 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3767 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3997 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 5002 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 548 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 549 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102_vm 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104 2024-05-08T22:10:00.616 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104_vm 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106_vm 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6110 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6111 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6112 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6113 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6117 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826_vm 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 827 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 828 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/12347: 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:total 655688 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 May 8 21:54 . 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 8 21:54 .. 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 112 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 113 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 8 21:54 12182 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12182_fsm 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12182_vm 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12184 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12186 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12187_fsm 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12187_vm 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12189 2024-05-08T22:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12191 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12192_fsm 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12192_vm 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12194 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12196 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12197_fsm 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12197_vm 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12199 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12201 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12202_fsm 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12202_vm 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12204 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12206 2024-05-08T22:10:00.619 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 12207_fsm 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12207_vm 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12209 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12211 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12212 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 12214 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 12216 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 8 21:54 1247 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1247_fsm 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1247_vm 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 8 21:54 1249 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1249_fsm 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1249_vm 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 8 21:54 1255 2024-05-08T22:10:00.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1255_fsm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1255_vm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 22:09 1259 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1259_fsm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1259_vm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1417_vm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 1418_vm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 11345920 May 8 22:09 16387 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 22:09 16387_fsm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 22:09 16387_vm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 6660096 May 8 22:09 16393 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 22:09 16393_fsm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 22:09 16393_vm 2024-05-08T22:10:00.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 470507520 May 8 22:09 16396 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 May 8 22:08 16396_fsm 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:57 16396_vm 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 8 22:09 16397 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 622592 May 8 22:09 16399 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 72638464 May 8 22:09 16401 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 101695488 May 8 22:09 16403 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 22:09 16403_fsm 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 174 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 175 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2187 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2224_vm 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328 2024-05-08T22:10:00.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2328_vm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2336_vm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2337 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2600 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2600_fsm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2600_vm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2601_fsm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2601_vm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2602 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2602_fsm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2602_vm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2603 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2603_fsm 2024-05-08T22:10:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2603_vm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2604_vm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2605 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2605_fsm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2605_vm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2606_fsm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2606_vm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2607_fsm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2607_vm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 May 8 21:54 2608 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2608_fsm 2024-05-08T22:10:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2608_vm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 8 21:54 2609 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2609_fsm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2609_vm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2610 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2610_fsm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2610_vm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2611_vm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2612_fsm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2612_vm 2024-05-08T22:10:00.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2613_vm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2615_fsm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2615_vm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2616_fsm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2616_vm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 8 21:54 2617 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2617_fsm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2617_vm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 8 21:54 2618 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2618_fsm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2618_vm 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 May 8 22:09 2619 2024-05-08T22:10:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:59 2619_fsm 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2619_vm 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2620_vm 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2650 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2651 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2652 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2653 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2654 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2655 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2656 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2657 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 8 21:54 2658 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2659 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2660 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2661 2024-05-08T22:10:00.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2662 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2663 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2664 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2665 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2666 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2667 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2668 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2669 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2670 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 8 21:54 2673 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 May 8 21:54 2674 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 8 21:54 2675 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2678 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2679 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2680 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2681 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2682 2024-05-08T22:10:00.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2683 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2684 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2685 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2686 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2687 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2688 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 8 21:54 2689 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 8 21:54 2690 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 8 21:54 2691 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2692 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2693 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:59 2696 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2699 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2701 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2702 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2703 2024-05-08T22:10:00.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 2704 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2753 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2753_fsm 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2753_vm 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2754 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2755 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2756 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 2757 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2830_vm 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2831 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2832_vm 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2833 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834 2024-05-08T22:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2834_vm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2835 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2836_vm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2837 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 8 21:54 2838 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2838_fsm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2838_vm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2839 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2840_fsm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2840_vm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2841 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2995_vm 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2996 2024-05-08T22:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3079_fsm 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3079_vm 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3080 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3081 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3085 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3118_vm 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3119 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3164 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3256_vm 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3257 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3258 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3350_vm 2024-05-08T22:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3351 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3379 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3380 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3381_vm 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3394 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3394_fsm 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3394_vm 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3395 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3439_vm 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3440 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 8 21:54 3455 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3456 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3456_fsm 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3456_vm 2024-05-08T22:10:00.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3466_vm 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3467 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3468 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3501_vm 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3502 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3503 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3534 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3541_fsm 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3541_vm 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3542 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3574 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3575 2024-05-08T22:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3576_vm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3596_vm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3597 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3598_vm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3599 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3600_fsm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3600_vm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3601_fsm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3601_vm 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602 2024-05-08T22:10:00.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3602_fsm 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3602_vm 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3603 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3603_fsm 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3603_vm 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3604 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3605 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3606 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3607 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3608 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 8 21:54 3609 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3712 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764 2024-05-08T22:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 3764_fsm 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3764_vm 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3766 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 3767 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3997 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 5002 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 548 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 549 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6102_vm 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6104_vm 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6106_vm 2024-05-08T22:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6110 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6111 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6112 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6113 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6117 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 826_vm 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 827 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 828 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 May 8 21:54 pg_internal.init 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 21:54 PG_VERSION 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-08T22:10:00.638 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 8 21:54 .. 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/global: 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:total 574 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 May 8 21:54 . 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1136 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1136_fsm 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1136_vm 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 1137 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1213 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1213_fsm 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1213_vm 2024-05-08T22:10:00.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1214 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1214_fsm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1214_vm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 1232 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 1233 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1260 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1260_fsm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1260_vm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1261 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1261_fsm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1261_vm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1262 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 1262_fsm 2024-05-08T22:10:00.640 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 1262_vm 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2396 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 8 21:54 2396_fsm 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2396_vm 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2397 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2671 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2672 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2676 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2677 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2694 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2695 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2697 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 8 21:54 2698 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2846 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2846_vm 2024-05-08T22:10:00.641 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2847 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2964 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2964_vm 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2965 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2966 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 2966_vm 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 2967 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3592 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 3592_vm 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 3593 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 4060 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 4060_vm 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 4061 2024-05-08T22:10:00.642 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6000 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6000_vm 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6001 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6002 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6100 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 6100_vm 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6114 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 6115 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 22:10 pg_control 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 8 21:54 pg_filenode.map 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 May 8 21:54 pg_internal.init 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/log: 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:total 3 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T22:10:00.643 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2808 May 8 22:10 postgresql-Wed.log 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-08T22:10:00.644 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 22:10 . 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 mappings 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 8 22:10 replorigin_checkpoint 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 snapshots 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 22:10 .. 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.645 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 22:10 .. 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 . 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 members 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 offsets 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 .. 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T22:10:00.646 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 21:54 .. 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:total 8 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 21:54 . 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 8 21:54 0000 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.647 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:total 13 2024-05-08T22:10:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 8 22:10 . 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 May 8 22:10 db_0.stat 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 May 8 22:10 db_12347.stat 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 May 8 22:10 global.stat 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 22:10 . 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:total 152 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 8 22:10 . 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 May 8 22:09 001D 2024-05-08T22:10:00.649 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:total 1048576 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 8 22:10 . 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.650 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000027 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000028 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000029 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000002A 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000002B 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000002C 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000002D 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000002E 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000002F 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000030 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000031 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000032 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000033 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000034 2024-05-08T22:10:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000035 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000036 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000037 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000038 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 000000010000000200000039 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000003A 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000003B 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:09 00000001000000020000003C 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:10 00000001000000020000003D 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 00000001000000020000003E 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:06 00000001000000020000003F 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000040 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:06 000000010000000200000041 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000042 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000043 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:06 000000010000000200000044 2024-05-08T22:10:00.652 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000045 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000046 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000047 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:06 000000010000000200000048 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000049 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 00000001000000020000004A 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 00000001000000020000004B 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 00000001000000020000004C 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:06 00000001000000020000004D 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000004E 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000004F 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000050 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000051 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000052 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000053 2024-05-08T22:10:00.653 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000054 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000055 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000056 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000057 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000058 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000059 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000005A 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000005B 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 00000001000000020000005C 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000005D 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000005E 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 00000001000000020000005F 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000060 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000061 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000062 2024-05-08T22:10:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000063 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000064 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:08 000000010000000200000065 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 8 22:07 000000010000000200000066 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 archive_status 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 8 21:54 . 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 8 22:10 .. 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:total 480 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 8 22:01 . 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 8 22:10 .. 2024-05-08T22:10:00.655 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 May 8 22:01 0000 2024-05-08T22:10:00.656 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 229376 May 8 22:09 0001 2024-05-08T22:10:00.657 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-08T22:10:00.668 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T22:10:00.676 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T22:10:00.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:10:00.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/2275646188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 msgr2=0x7fa6a0108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:00.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/2275646188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a0108fb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa69800b0a0 tx=0x7fa69800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:10:00.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/2275646188 shutdown_connections 2024-05-08T22:10:00.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/2275646188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:00.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/2275646188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a0108fb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:00.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/2275646188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0104210 0x7fa6a0106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/2275646188 >> 172.21.15.64:0/2275646188 conn(0x7fa6a00fb2f0 msgr2=0x7fa6a00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/2275646188 shutdown_connections 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.848+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/2275646188 wait complete. 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 Processor -- start 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 -- start start 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0104210 0x7fa6a00ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a00ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0100570 con 0x7fa6a0106b40 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01006e0 con 0x7fa6a01018e0 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a6bc7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01a0ee0 con 0x7fa6a0104210 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a5164700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a00ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:00.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a4963700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a4963700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.64:49532/0 (socket says 172.21.15.64:49532) 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a5164700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a00ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48706/0 (socket says 172.21.15.64:48706) 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa6a4963700 1 -- 172.21.15.64:0/3381059497 learned_addr learned my addr 172.21.15.64:0/3381059497 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.849+0000 7fa69ffff700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0104210 0x7fa6a00ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a4963700 1 -- 172.21.15.64:0/3381059497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0104210 msgr2=0x7fa6a00ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a4963700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0104210 0x7fa6a00ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a4963700 1 -- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 msgr2=0x7fa6a00ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a4963700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a00ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a4963700 1 -- 172.21.15.64:0/3381059497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa698009d00 con 0x7fa6a01018e0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a5164700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a4963700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0101200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d950 tx=0x7fa69400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69400f890 con 0x7fa6a01018e0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a01a1170 con 0x7fa6a01018e0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa694010460 con 0x7fa6a01018e0 2024-05-08T22:10:00.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940195c0 con 0x7fa6a01018e0 2024-05-08T22:10:00.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.850+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01a16d0 con 0x7fa6a01018e0 2024-05-08T22:10:00.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.851+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a00fce70 con 0x7fa6a01018e0 2024-05-08T22:10:00.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.852+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa6940108d0 con 0x7fa6a01018e0 2024-05-08T22:10:00.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.853+0000 7fa69dffb700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6900a1120 0x7fa6900a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:00.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.853+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa6940cc650 con 0x7fa6a01018e0 2024-05-08T22:10:00.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.853+0000 7fa69ffff700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6900a1120 0x7fa6900a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:00.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.853+0000 7fa69ffff700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6900a1120 0x7fa6900a35e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009c60 tx=0x7fa68c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:00.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:00.857+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa694095df0 con 0x7fa6a01018e0 2024-05-08T22:10:01.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:00 smithi160 ceph-mon[46962]: overall HEALTH_OK 2024-05-08T22:10:01.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.068+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6a0067c10 con 0x7fa6a01018e0 2024-05-08T22:10:01.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.070+0000 7fa69dffb700 1 -- 172.21.15.64:0/3381059497 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22495 (secure 0 0 0) 0x7fa694016090 con 0x7fa6a01018e0 2024-05-08T22:10:01.071 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:01.071 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":81,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:54:21.242448+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:10:01.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6900a1120 msgr2=0x7fa6900a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6900a1120 0x7fa6900a35e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009c60 tx=0x7fa68c009260 comp rx=0 tx=0).stop 2024-05-08T22:10:01.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 msgr2=0x7fa6a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0101200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d950 tx=0x7fa69400dc60 comp rx=0 tx=0).stop 2024-05-08T22:10:01.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 shutdown_connections 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa6900a1120 0x7fa6900a35e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a01018e0 0x7fa6a0101200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a0106b40 0x7fa6a00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 --2- 172.21.15.64:0/3381059497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0104210 0x7fa6a00ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 >> 172.21.15.64:0/3381059497 conn(0x7fa6a00fb2f0 msgr2=0x7fa6a0102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 shutdown_connections 2024-05-08T22:10:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.072+0000 7fa6a6bc7700 1 -- 172.21.15.64:0/3381059497 wait complete. 2024-05-08T22:10:01.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:00 smithi064 ceph-mon[42500]: overall HEALTH_OK 2024-05-08T22:10:01.086 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:53:21.578075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:53:22.933075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:10:01.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:10:01.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:00 smithi186 ceph-mon[46619]: overall HEALTH_OK 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 -- 172.21.15.64:0/3501447429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 msgr2=0x7f34e4103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 --2- 172.21.15.64:0/3501447429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4103e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009990 tx=0x7f34d4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 -- 172.21.15.64:0/3501447429 shutdown_connections 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 --2- 172.21.15.64:0/3501447429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4106ca0 0x7f34e41090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 --2- 172.21.15.64:0/3501447429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4103e30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 --2- 172.21.15.64:0/3501447429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34e4104370 0x7f34e4106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 -- 172.21.15.64:0/3501447429 >> 172.21.15.64:0/3501447429 conn(0x7f34e40fb3f0 msgr2=0x7f34e40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 -- 172.21.15.64:0/3501447429 shutdown_connections 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.218+0000 7f34e9507700 1 -- 172.21.15.64:0/3501447429 wait complete. 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 Processor -- start 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 -- start start 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 0x7f34e41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34e4106ca0 0x7f34e419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e419e9c0 con 0x7f34e4104370 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e419eb00 con 0x7f34e4106ca0 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e9507700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e419ec90 con 0x7f34e4101a40 2024-05-08T22:10:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 0x7f34e41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 0x7f34e41973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48714/0 (socket says 172.21.15.64:48714) 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e27fc700 1 -- 172.21.15.64:0/964818191 learned_addr learned my addr 172.21.15.64:0/964818191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e2ffd700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.219+0000 7f34e37fe700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34e4106ca0 0x7f34e419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e27fc700 1 -- 172.21.15.64:0/964818191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 msgr2=0x7f34e4196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e27fc700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e27fc700 1 -- 172.21.15.64:0/964818191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34e4106ca0 msgr2=0x7f34e419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e27fc700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34e4106ca0 0x7f34e419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e27fc700 1 -- 172.21.15.64:0/964818191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d40095f0 con 0x7f34e4104370 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e2ffd700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e27fc700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 0x7f34e41973d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9a0 tx=0x7f34d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8009790 con 0x7f34e4104370 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e419ef70 con 0x7f34e4104370 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d8010430 con 0x7f34e4104370 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8018680 con 0x7f34e4104370 2024-05-08T22:10:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.220+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e419f3f0 con 0x7f34e4104370 2024-05-08T22:10:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.221+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40fcf70 con 0x7f34e4104370 2024-05-08T22:10:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.222+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34d80105a0 con 0x7f34e4104370 2024-05-08T22:10:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.223+0000 7f34cbfff700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34cc0a1120 0x7f34cc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.223+0000 7f34e2ffd700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34cc0a1120 0x7f34cc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.223+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34d8014070 con 0x7f34e4104370 2024-05-08T22:10:01.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.223+0000 7f34e2ffd700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34cc0a1120 0x7f34cc0a35e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b2e0 tx=0x7f34d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:01.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.226+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f34d8096410 con 0x7f34e4104370 2024-05-08T22:10:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.430+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34e410c740 con 0x7f34e4104370 2024-05-08T22:10:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.431+0000 7f34cbfff700 1 -- 172.21.15.64:0/964818191 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34d80260a0 con 0x7f34e4104370 2024-05-08T22:10:01.432 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T22:10:01.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34cc0a1120 msgr2=0x7f34cc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34cc0a1120 0x7f34cc0a35e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b2e0 tx=0x7f34d4005fd0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 msgr2=0x7f34e41973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 0x7f34e41973d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9a0 tx=0x7f34d800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 shutdown_connections 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34cc0a1120 0x7f34cc0a35e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34e4106ca0 0x7f34e419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e4101a40 0x7f34e4196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 --2- 172.21.15.64:0/964818191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e4104370 0x7f34e41973d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 >> 172.21.15.64:0/964818191 conn(0x7f34e40fb3f0 msgr2=0x7f34e41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 shutdown_connections 2024-05-08T22:10:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.433+0000 7f34e9507700 1 -- 172.21.15.64:0/964818191 wait complete. 2024-05-08T22:10:01.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:10:01.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 -- 172.21.15.64:0/634528995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc100fd0 msgr2=0x7f94fc101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 --2- 172.21.15.64:0/634528995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc100fd0 0x7f94fc101450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009990 tx=0x7f94f0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 -- 172.21.15.64:0/634528995 shutdown_connections 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 --2- 172.21.15.64:0/634528995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94fc0ffe80 0x7f94fc1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 --2- 172.21.15.64:0/634528995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94fc101990 0x7f94fc1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 --2- 172.21.15.64:0/634528995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc100fd0 0x7f94fc101450 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 -- 172.21.15.64:0/634528995 >> 172.21.15.64:0/634528995 conn(0x7f94fc0fb3f0 msgr2=0x7f94fc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 -- 172.21.15.64:0/634528995 shutdown_connections 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.576+0000 7f950115d700 1 -- 172.21.15.64:0/634528995 wait complete. 2024-05-08T22:10:01.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 Processor -- start 2024-05-08T22:10:01.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 -- start start 2024-05-08T22:10:01.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94fc0ffe80 0x7f94fc196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94fc100fd0 0x7f94fc197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 0x7f94fc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19e9f0 con 0x7f94fc101990 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19eb30 con 0x7f94fc100fd0 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.577+0000 7f950115d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19ecc0 con 0x7f94fc0ffe80 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 0x7f94fc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 0x7f94fc19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48730/0 (socket says 172.21.15.64:48730) 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 -- 172.21.15.64:0/4098725362 learned_addr learned my addr 172.21.15.64:0/4098725362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fa59c700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94fc100fd0 0x7f94fc197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fad9d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94fc0ffe80 0x7f94fc196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 -- 172.21.15.64:0/4098725362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94fc0ffe80 msgr2=0x7f94fc196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94fc0ffe80 0x7f94fc196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 -- 172.21.15.64:0/4098725362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94fc100fd0 msgr2=0x7f94fc197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94fc100fd0 0x7f94fc197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 -- 172.21.15.64:0/4098725362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00095f0 con 0x7f94fc101990 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94fb59e700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 0x7f94fc19e420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f94f4007ee0 tx=0x7f94f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4017070 con 0x7f94fc101990 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc19efa0 con 0x7f94fc101990 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f400f040 con 0x7f94fc101990 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4005580 con 0x7f94fc101990 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.578+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc19f420 con 0x7f94fc101990 2024-05-08T22:10:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.579+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc067460 con 0x7f94fc101990 2024-05-08T22:10:01.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.580+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f94f400db20 con 0x7f94fc101990 2024-05-08T22:10:01.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.581+0000 7f94e3fff700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f94e40a1120 0x7f94e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.581+0000 7f94fad9d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f94e40a1120 0x7f94e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.581+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f94f40cc630 con 0x7f94fc101990 2024-05-08T22:10:01.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.581+0000 7f94fad9d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f94e40a1120 0x7f94e40a35e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005950 tx=0x7f94ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:01.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.584+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f94f4095dd0 con 0x7f94fc101990 2024-05-08T22:10:01.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.795+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94fc10b770 con 0x7f94fc101990 2024-05-08T22:10:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.801+0000 7f94e3fff700 1 -- 172.21.15.64:0/4098725362 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94f40993f0 con 0x7f94fc101990 2024-05-08T22:10:01.802 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-08T22:10:01.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.803+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f94e40a1120 msgr2=0x7f94e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.803+0000 7f950115d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f94e40a1120 0x7f94e40a35e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005950 tx=0x7f94ec00b410 comp rx=0 tx=0).stop 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 msgr2=0x7f94fc19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 0x7f94fc19e420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f94f4007ee0 tx=0x7f94f400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 shutdown_connections 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f94e40a1120 0x7f94e40a35e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94fc101990 0x7f94fc19e420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94fc0ffe80 0x7f94fc196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 --2- 172.21.15.64:0/4098725362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94fc100fd0 0x7f94fc197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 >> 172.21.15.64:0/4098725362 conn(0x7f94fc0fb3f0 msgr2=0x7f94fc0692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 shutdown_connections 2024-05-08T22:10:01.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.804+0000 7f950115d700 1 -- 172.21.15.64:0/4098725362 wait complete. 2024-05-08T22:10:01.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:10:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 -- 172.21.15.64:0/1686732632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8100f20 msgr2=0x7faea8101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 --2- 172.21.15.64:0/1686732632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8100f20 0x7faea8101360 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fae90009990 tx=0x7fae90009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 -- 172.21.15.64:0/1686732632 shutdown_connections 2024-05-08T22:10:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 --2- 172.21.15.64:0/1686732632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 0x7faea81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 --2- 172.21.15.64:0/1686732632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8100f20 0x7faea8101360 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 --2- 172.21.15.64:0/1686732632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea81020f0 0x7faea8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 -- 172.21.15.64:0/1686732632 >> 172.21.15.64:0/1686732632 conn(0x7faea80fc4e0 msgr2=0x7faea80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 -- 172.21.15.64:0/1686732632 shutdown_connections 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.946+0000 7faead907700 1 -- 172.21.15.64:0/1686732632 wait complete. 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 Processor -- start 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 -- start start 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea8100f20 0x7faea8196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 0x7faea8197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 0x7faea819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea819ea00 con 0x7faea81020f0 2024-05-08T22:10:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea819eb40 con 0x7faea8100f20 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faead907700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea819ecd0 con 0x7faea8102ab0 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 0x7faea8197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 0x7faea8197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48740/0 (socket says 172.21.15.64:48740) 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 -- 172.21.15.64:0/1175914528 learned_addr learned my addr 172.21.15.64:0/1175914528 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea6ffd700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea8100f20 0x7faea8196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea77fe700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 0x7faea819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 -- 172.21.15.64:0/1175914528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 msgr2=0x7faea819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 0x7faea819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 -- 172.21.15.64:0/1175914528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea8100f20 msgr2=0x7faea8196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea8100f20 0x7faea8196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.947+0000 7faea67fc700 1 -- 172.21.15.64:0/1175914528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900095f0 con 0x7faea81020f0 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faea77fe700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 0x7faea819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:01.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faea6ffd700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea8100f20 0x7faea8196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:10:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faea67fc700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 0x7faea8197410 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9a0 tx=0x7fae9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98009790 con 0x7faea81020f0 2024-05-08T22:10:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea819efb0 con 0x7faea81020f0 2024-05-08T22:10:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae9800cbd0 con 0x7faea81020f0 2024-05-08T22:10:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98018680 con 0x7faea81020f0 2024-05-08T22:10:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.948+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea819f480 con 0x7faea81020f0 2024-05-08T22:10:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.949+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea8108460 con 0x7faea81020f0 2024-05-08T22:10:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.950+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fae9800cd40 con 0x7faea81020f0 2024-05-08T22:10:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.951+0000 7faeac905700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fae940a1120 0x7fae940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.951+0000 7faea6ffd700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fae940a1120 0x7fae940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.951+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae98014070 con 0x7faea81020f0 2024-05-08T22:10:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.951+0000 7faea6ffd700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fae940a1120 0x7fae940a35e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b2e0 tx=0x7fae9000b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:01.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:01.955+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae98052d60 con 0x7faea81020f0 2024-05-08T22:10:02.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:01 smithi160 ceph-mon[46962]: pgmap v721: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.51k op/s 2024-05-08T22:10:02.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:01 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3381059497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:02.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:01 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/964818191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:10:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:01 smithi186 ceph-mon[46619]: pgmap v721: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.51k op/s 2024-05-08T22:10:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:01 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3381059497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:02.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:01 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/964818191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:10:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.165+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faea802cdf0 con 0x7faea81020f0 2024-05-08T22:10:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.166+0000 7faeac905700 1 -- 172.21.15.64:0/1175914528 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae980962d0 con 0x7faea81020f0 2024-05-08T22:10:02.167 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fae940a1120 msgr2=0x7fae940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fae940a1120 0x7fae940a35e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b2e0 tx=0x7fae9000b6f0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 msgr2=0x7faea8197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 0x7faea8197410 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9a0 tx=0x7fae9800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 shutdown_connections 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fae940a1120 0x7fae940a35e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faea8100f20 0x7faea8196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faea8102ab0 0x7faea819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 --2- 172.21.15.64:0/1175914528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea81020f0 0x7faea8197410 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 >> 172.21.15.64:0/1175914528 conn(0x7faea80fc4e0 msgr2=0x7faea81055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 shutdown_connections 2024-05-08T22:10:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.168+0000 7faead907700 1 -- 172.21.15.64:0/1175914528 wait complete. 2024-05-08T22:10:02.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:01 smithi064 ceph-mon[42500]: pgmap v721: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.51k op/s 2024-05-08T22:10:02.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:01 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3381059497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:02.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:01 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/964818191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:10:02.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 -- 172.21.15.64:0/3239795308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4078100f50 msgr2=0x7f40781013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 --2- 172.21.15.64:0/3239795308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4078100f50 0x7f40781013b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f406c0099c0 tx=0x7f406c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 -- 172.21.15.64:0/3239795308 shutdown_connections 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 --2- 172.21.15.64:0/3239795308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40780ffd60 0x7f4078100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 --2- 172.21.15.64:0/3239795308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40781018f0 0x7f40781081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 --2- 172.21.15.64:0/3239795308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4078100f50 0x7f40781013b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 -- 172.21.15.64:0/3239795308 >> 172.21.15.64:0/3239795308 conn(0x7f40780fb2f0 msgr2=0x7f40780fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 -- 172.21.15.64:0/3239795308 shutdown_connections 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.307+0000 7f407e42b700 1 -- 172.21.15.64:0/3239795308 wait complete. 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.308+0000 7f407e42b700 1 Processor -- start 2024-05-08T22:10:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.308+0000 7f407e42b700 1 -- start start 2024-05-08T22:10:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407e42b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 0x7f4078196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407e42b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4078100f50 0x7f4078197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407e42b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40781018f0 0x7f407819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407e42b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819e950 con 0x7f40780ffd60 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407e42b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819ea90 con 0x7f4078100f50 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407e42b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819ec20 con 0x7f40781018f0 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 0x7f4078196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 0x7f4078196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48764/0 (socket says 172.21.15.64:48764) 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 -- 172.21.15.64:0/2701662543 learned_addr learned my addr 172.21.15.64:0/2701662543 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f40777fe700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4078100f50 0x7f4078197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f407c9c8700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40781018f0 0x7f407819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 -- 172.21.15.64:0/2701662543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40781018f0 msgr2=0x7f407819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40781018f0 0x7f407819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 -- 172.21.15.64:0/2701662543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4078100f50 msgr2=0x7f4078197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4078100f50 0x7f4078197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 -- 172.21.15.64:0/2701662543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0095f0 con 0x7f40780ffd60 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f4077fff700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 0x7f4078196e20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f406800e970 tx=0x7f406800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.309+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068009790 con 0x7f40780ffd60 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.310+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f406800c7e0 con 0x7f40780ffd60 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.310+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407819ef00 con 0x7f40780ffd60 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.310+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068003c70 con 0x7f40780ffd60 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.310+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407819f320 con 0x7f40780ffd60 2024-05-08T22:10:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.310+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078067c10 con 0x7f40780ffd60 2024-05-08T22:10:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.312+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f406800cc50 con 0x7f40780ffd60 2024-05-08T22:10:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.312+0000 7f40757fa700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f40600a1120 0x7f40600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.312+0000 7f40777fe700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f40600a1120 0x7f40600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.312+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4068014070 con 0x7f40780ffd60 2024-05-08T22:10:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.313+0000 7f40777fe700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f40600a1120 0x7f40600a35e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f406c00b580 tx=0x7f406c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.316+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4068096320 con 0x7f40780ffd60 2024-05-08T22:10:02.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.588+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f407810b820 con 0x7f40780ffd60 2024-05-08T22:10:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.597+0000 7f40757fa700 1 -- 172.21.15.64:0/2701662543 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733410 (secure 0 0 0) 0x7f4068003dd0 con 0x7f40780ffd60 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 18, 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 14152, 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "x", 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2024-05-08T22:10:02.609 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6800", 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3946872331 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6801", 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3946872331 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.610 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6801/3946872331", 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2024-05-08T21:48:20.034572+0000", 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540138322906710015, 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24112, 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "z", 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540138322906710015, 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.612 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.615 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.628 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-05-08T22:10:02.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-05-08T22:10:02.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.631 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-05-08T22:10:02.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.636 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.637 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:10:02.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:10:02.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:10:02.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:10:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-05-08T22:10:02.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-05-08T22:10:02.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-05-08T22:10:02.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-05-08T22:10:02.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-05-08T22:10:02.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:10:02.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:10:02.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-05-08T22:10:02.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.687 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-05-08T22:10:02.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.698 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:10:02.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-05-08T22:10:02.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-05-08T22:10:02.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-05-08T22:10:02.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:10:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:10:02.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.711 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:10:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:10:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:10:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:10:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:10:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:10:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:10:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:10:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:10:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:10:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:10:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-08T22:10:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-05-08T22:10:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-05-08T22:10:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-05-08T22:10:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-05-08T22:10:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-05-08T22:10:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-05-08T22:10:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:10:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-05-08T22:10:02.823 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.824 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.825 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.831 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.834 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-05-08T22:10:02.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.838 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.840 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.842 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.846 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.848 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.850 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-05-08T22:10:02.852 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.853 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.854 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.855 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-05-08T22:10:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-05-08T22:10:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-05-08T22:10:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:10:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-05-08T22:10:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:10:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:10:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-05-08T22:10:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-05-08T22:10:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24113, 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "y", 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540138322906710015, 2024-05-08T22:10:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-05-08T22:10:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-05-08T22:10:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-05-08T22:10:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-05-08T22:10:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-05-08T22:10:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-05-08T22:10:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-05-08T22:10:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-08T22:10:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-05-08T22:10:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-08T22:10:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-05-08T22:10:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:10:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:10:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-05-08T22:10:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-05-08T22:10:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-05-08T22:10:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-05-08T22:10:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:10:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-05-08T22:10:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-05-08T22:10:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:10:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-05-08T22:10:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-05-08T22:10:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-05-08T22:10:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:10:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-05-08T22:10:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:10:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:10:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:10:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:10:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-05-08T22:10:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:10:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:10:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:10:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:10:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-05-08T22:10:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:10:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:10:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:10:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:10:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:10:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-08T22:10:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-08T22:10:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-05-08T22:10:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-05-08T22:10:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-05-08T22:10:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-05-08T22:10:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-05-08T22:10:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-05-08T22:10:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:10:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:10:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-05-08T22:10:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-05-08T22:10:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-05-08T22:10:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-05-08T22:10:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-05-08T22:10:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-05-08T22:10:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-05-08T22:10:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-05-08T22:10:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-08T22:10:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-08T22:10:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:10:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-05-08T22:10:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-05-08T22:10:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-05-08T22:10:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-05-08T22:10:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-08T22:10:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-05-08T22:10:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-05-08T22:10:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-05-08T22:10:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-05-08T22:10:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2024-05-08T22:10:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "restful" 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.480 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.481 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-05-08T22:10:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-05-08T22:10:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-05-08T22:10:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:10:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:10:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-05-08T22:10:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-05-08T22:10:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-05-08T22:10:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-05-08T22:10:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:10:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-05-08T22:10:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-05-08T22:10:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:10:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-05-08T22:10:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-05-08T22:10:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-08T22:10:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-05-08T22:10:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:10:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:10:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:10:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-05-08T22:10:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-05-08T22:10:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:10:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-05-08T22:10:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-05-08T22:10:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:10:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:10:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:10:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:10:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:10:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:10:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:10:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:10:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:10:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:10:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:10:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:10:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:10:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-05-08T22:10:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-05-08T22:10:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-05-08T22:10:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:10:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:10:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:10:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:10:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-05-08T22:10:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-05-08T22:10:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-05-08T22:10:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:10:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:10:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:10:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:10:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-05-08T22:10:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:10:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-05-08T22:10:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-05-08T22:10:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-08T22:10:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-08T22:10:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:10:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-05-08T22:10:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-05-08T22:10:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-08T22:10:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:10:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-08T22:10:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-08T22:10:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-08T22:10:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "services": { 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard": "https://172.21.15.64:8443/" 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-08T22:10:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-08T22:10:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-08T22:10:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 3, 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-08T22:10:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1922858447 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3517323888 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1958414101 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1577677719 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-08T22:10:03.931 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T22:10:03.933 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/4098725362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:10:03.933 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1175914528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:10:03.933 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:02 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2701662543' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:10:03.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/4098725362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:10:03.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1175914528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:10:03.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:02 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2701662543' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:10:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.601+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f40600a1120 msgr2=0x7f40600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.601+0000 7f407e42b700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f40600a1120 0x7f40600a35e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f406c00b580 tx=0x7f406c005fb0 comp rx=0 tx=0).stop 2024-05-08T22:10:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.601+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 msgr2=0x7f4078196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.601+0000 7f407e42b700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 0x7f4078196e20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f406800e970 tx=0x7f406800ec80 comp rx=0 tx=0).stop 2024-05-08T22:10:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 shutdown_connections 2024-05-08T22:10:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f40600a1120 0x7f40600a35e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40781018f0 0x7f407819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40780ffd60 0x7f4078196e20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 --2- 172.21.15.64:0/2701662543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4078100f50 0x7f4078197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 >> 172.21.15.64:0/2701662543 conn(0x7f40780fb2f0 msgr2=0x7f40780fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 shutdown_connections 2024-05-08T22:10:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:02.602+0000 7f407e42b700 1 -- 172.21.15.64:0/2701662543 wait complete. 2024-05-08T22:10:03.941 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T22:10:03.941 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T22:10:03.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/4098725362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:10:03.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1175914528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:10:03.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:02 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2701662543' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:10:04.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:03 smithi186 ceph-mon[46619]: pgmap v722: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.78k op/s 2024-05-08T22:10:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:10:04.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:03 smithi064 ceph-mon[42500]: pgmap v722: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.78k op/s 2024-05-08T22:10:04.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:03 smithi160 ceph-mon[46962]: pgmap v722: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.78k op/s 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2227, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2229, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2419324, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 692598, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.189733160, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.052764094, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.053677844, 2024-05-08T22:10:05.297 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.066794413, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2227, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.414822493, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000186269 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:05.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 927, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1129, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 230616, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 720442, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.094308805, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.029937941, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.020544930, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.036107759, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1128, 2024-05-08T22:10:05.299 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.206096396, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000182709 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 433632 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 301, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 278, 2024-05-08T22:10:05.300 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 624396, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 155783, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.039209571, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.014481944, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.018186766, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.006540684, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 277, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.067701711, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000244410 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.301 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 624128, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 155472 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-08T22:10:05.302 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5242, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.145610571, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000027777 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-05-08T22:10:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:05.304 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 471, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 166, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 305, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 234, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 23, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2024-05-08T22:10:05.305 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:05.306 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:10:05.307 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-05-08T22:10:05.308 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-05-08T22:10:05.309 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 829, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 1, 2024-05-08T22:10:05.310 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 5961235, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 5960636, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 829 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 459, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 461, 2024-05-08T22:10:05.311 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 2, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 52, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 52, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 0, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 471, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 471, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 0, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 50720, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 207156 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-05-08T22:10:05.312 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.314 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.315 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.316 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.320 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.321 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-05-08T22:10:05.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.323 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 1, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 0, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 1, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 2351, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.324 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:05.326 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 206124, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 503, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:05.327 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 158, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 1416622, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 11624, 2024-05-08T22:10:05.328 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 3150, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 21, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 3, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 3150, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3150, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 9.967812703, 2024-05-08T22:10:05.329 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003164384 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3150, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3151 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 3150, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 3150, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 1905, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2024-05-08T22:10:05.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 1243, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 1, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:05.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-05-08T22:10:05.332 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-05-08T22:10:05.333 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 81, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 2, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 5, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 7, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 2, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2024-05-08T22:10:05.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-05-08T22:10:05.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 3451, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 2945384, 2024-05-08T22:10:05.336 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 3451, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3451, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 2945384, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:05.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3150, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 1766936, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3150, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1766936, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3150, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 3150, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3150, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 3150, 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:05.339 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:05.340 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55df6fe2e5a0": { 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:05.341 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:05.342 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T22:10:05.685 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-08T22:10:05.685 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-08T22:10:05.686 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-08T22:10:05.686 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-08T22:10:05.686 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T22:10:05.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T22:10:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:10:05.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:05 smithi064 ceph-mon[42500]: pgmap v723: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.92k op/s 2024-05-08T22:10:06.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:05 smithi186 ceph-mon[46619]: pgmap v723: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.92k op/s 2024-05-08T22:10:06.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:05 smithi160 ceph-mon[46962]: pgmap v723: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.92k op/s 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001754979, 2024-05-08T22:10:06.907 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000041695, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000591379, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000754984, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000251661 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T22:10:06.908 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 209, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 189392, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.013908208, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.010609097, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001610279, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 207, 2024-05-08T22:10:06.909 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.059295038, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000286449 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 189392 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 276, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 273, 2024-05-08T22:10:06.910 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 488528, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 156064, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.037872420, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.014608152, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.015987029, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.006647216, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 272, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.070403072, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000258834 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:06.911 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 488528, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 156064 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:06.912 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:06.913 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:06.914 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 147920, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 27, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-08T22:10:06.915 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 278, 2024-05-08T22:10:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 449509, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 278, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 278, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 449509, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:06.917 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T22:10:06.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:06 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:07.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:06 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:07.263 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T22:10:07.263 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T22:10:07.264 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-08T22:10:07.264 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-08T22:10:07.264 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T22:10:07.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T22:10:07.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:06 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:07.495 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:10:07.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:07 smithi064 ceph-mon[42500]: pgmap v724: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.80k op/s 2024-05-08T22:10:08.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:07 smithi186 ceph-mon[46619]: pgmap v724: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.80k op/s 2024-05-08T22:10:08.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:07 smithi160 ceph-mon[46962]: pgmap v724: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.80k op/s 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 983, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 985, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 330233, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 305714, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 7, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.106186239, 2024-05-08T22:10:08.513 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.035260905, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.027311864, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.034018460, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 983, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.202795231, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000206302 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.514 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2368, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2342, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2321711, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 797851, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.179405506, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.049042419, 2024-05-08T22:10:08.515 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.055299709, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.065405174, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2342, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.358370351, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000153018 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 633424, 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 156512 2024-05-08T22:10:08.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 5, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 210, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1668, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 519904, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 7, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.014935902, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.010533313, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002545750, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000065711, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 208, 2024-05-08T22:10:08.517 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.062124967, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000298677 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 518960 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-05-08T22:10:08.518 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5028, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.125676200, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000024995 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.519 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-05-08T22:10:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 377, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 58, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 319, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 294, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 11, 2024-05-08T22:10:08.521 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2024-05-08T22:10:08.522 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2024-05-08T22:10:08.522 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-05-08T22:10:08.522 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-05-08T22:10:08.522 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:08.522 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-05-08T22:10:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:08.525 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:08.525 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:08.525 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:08.525 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2024-05-08T22:10:08.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:08.527 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-05-08T22:10:08.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 756, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 1, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-05-08T22:10:08.529 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 5359563, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 5358989, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.530 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 756 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 376, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 376, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 0, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 18, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 18, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 0, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 377, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 377, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 0, 2024-05-08T22:10:08.531 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 50636, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 207156 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-05-08T22:10:08.532 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.533 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-05-08T22:10:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.536 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.537 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-05-08T22:10:08.539 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.540 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:08.541 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.542 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.544 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-05-08T22:10:08.545 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 2, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 1, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 3714, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:08.546 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:08.547 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:08.548 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 211924, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 425, 2024-05-08T22:10:08.549 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 158, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:08.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 1120367, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 9609, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-05-08T22:10:08.551 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 3048, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 21, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 3, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 3048, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3048, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 17.126678397, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.005618988 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3048, 2024-05-08T22:10:08.552 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3049 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 3048, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 3048, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 1886, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 1160, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2024-05-08T22:10:08.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 1, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:08.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-05-08T22:10:08.555 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-05-08T22:10:08.556 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 81, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 2, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 6, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 8, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 2, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-05-08T22:10:08.557 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-05-08T22:10:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 3352, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 2333422, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 3352, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:08.559 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3352, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 2333422, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:08.560 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3048, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 1165264, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3048, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1165264, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.561 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3048, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 3048, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3048, 2024-05-08T22:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 3048, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:08.563 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x5585f677c5a0": { 2024-05-08T22:10:08.564 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:08.565 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:08.566 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:08.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:08.566 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T22:10:08.898 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-08T22:10:08.898 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-08T22:10:08.898 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-08T22:10:08.898 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-08T22:10:08.898 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T22:10:08.898 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T22:10:09.131 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001548509, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T22:10:10.120 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000384852, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.121 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 282, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 277, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 492064, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 157696, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.038776393, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.014641634, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.017606988, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.006617367, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 277, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.073045514, 2024-05-08T22:10:10.122 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000263702 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 492064, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 157696 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 210, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:10:10.123 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 189568, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.014009789, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.010605292, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001752994, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 208, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.060567996, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000291192 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:10.124 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 189568 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:10.125 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:10.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:10.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148744, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:10.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-08T22:10:10.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 280, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 449769, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 280, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 280, 2024-05-08T22:10:10.130 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 449769, 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-08T22:10:10.131 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-08T22:10:10.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:09 smithi186 ceph-mon[46619]: pgmap v725: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-05-08T22:10:10.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:09 smithi064 ceph-mon[42500]: pgmap v725: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-05-08T22:10:10.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:09 smithi160 ceph-mon[46962]: pgmap v725: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-05-08T22:10:10.505 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T22:10:10.505 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T22:10:10.505 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-08T22:10:10.505 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-08T22:10:10.505 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T22:10:10.505 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T22:10:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:10:11.705 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-08T22:10:11.706 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:11.706 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 5765, 2024-05-08T22:10:11.706 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 7471, 2024-05-08T22:10:11.707 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1921711, 2024-05-08T22:10:11.707 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 11376626, 2024-05-08T22:10:11.707 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 10, 2024-05-08T22:10:11.707 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:11.707 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.436458460, 2024-05-08T22:10:11.707 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.174466297, 2024-05-08T22:10:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.186492128, 2024-05-08T22:10:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.079520633, 2024-05-08T22:10:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 7471, 2024-05-08T22:10:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2.122075586, 2024-05-08T22:10:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000284041 2024-05-08T22:10:11.709 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.709 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:11.709 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 8724, 2024-05-08T22:10:11.709 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001882979, 2024-05-08T22:10:11.709 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000215 2024-05-08T22:10:11.710 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.710 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:11.710 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:11.710 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.710 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:11.711 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 6328, 2024-05-08T22:10:11.711 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 8364, 2024-05-08T22:10:11.711 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1838536, 2024-05-08T22:10:11.711 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 7280445, 2024-05-08T22:10:11.711 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 153, 2024-05-08T22:10:11.711 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 9, 2024-05-08T22:10:11.712 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.458995335, 2024-05-08T22:10:11.712 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.144112225, 2024-05-08T22:10:11.712 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.304348014, 2024-05-08T22:10:11.712 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.082066613, 2024-05-08T22:10:11.712 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:11.712 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 8361, 2024-05-08T22:10:11.713 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.033342183, 2024-05-08T22:10:11.713 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000123590 2024-05-08T22:10:11.713 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.713 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:11.713 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 11591, 2024-05-08T22:10:11.713 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001719129, 2024-05-08T22:10:11.714 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000148 2024-05-08T22:10:11.714 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.714 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 58672, 2024-05-08T22:10:11.714 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 475616 2024-05-08T22:10:11.714 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.714 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:11.715 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 10328, 2024-05-08T22:10:11.715 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 9346, 2024-05-08T22:10:11.715 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3919492, 2024-05-08T22:10:11.715 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 3875637, 2024-05-08T22:10:11.715 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 9, 2024-05-08T22:10:11.715 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 7, 2024-05-08T22:10:11.716 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.559753755, 2024-05-08T22:10:11.716 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.191021492, 2024-05-08T22:10:11.716 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.382258896, 2024-05-08T22:10:11.716 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.066125790, 2024-05-08T22:10:11.716 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:11.716 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 9345, 2024-05-08T22:10:11.717 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.475955094, 2024-05-08T22:10:11.717 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000157940 2024-05-08T22:10:11.717 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.717 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:11.717 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 16634, 2024-05-08T22:10:11.717 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003600118, 2024-05-08T22:10:11.718 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000216 2024-05-08T22:10:11.718 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.718 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 670464, 2024-05-08T22:10:11.718 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 163840 2024-05-08T22:10:11.718 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:11.719 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:11.720 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3077, 2024-05-08T22:10:11.720 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.487904917, 2024-05-08T22:10:11.720 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000483557 2024-05-08T22:10:11.720 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.720 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.720 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 132, 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.004093565, 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000031011 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.721 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.722 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-08T22:10:11.722 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 2389, 2024-05-08T22:10:11.722 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 2010, 2024-05-08T22:10:11.722 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-08T22:10:11.722 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2010, 2024-05-08T22:10:11.722 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 5.931420779, 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002950955 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 379, 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:11.723 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 609, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 3, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 43, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 11, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 19, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 13, 2024-05-08T22:10:11.724 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 20, 2024-05-08T22:10:11.725 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 1338, 2024-05-08T22:10:11.725 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 4, 2024-05-08T22:10:11.725 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 4, 2024-05-08T22:10:11.725 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 11, 2024-05-08T22:10:11.725 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 3218, 2024-05-08T22:10:11.725 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 2109, 2024-05-08T22:10:11.726 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 364, 2024-05-08T22:10:11.726 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 102, 2024-05-08T22:10:11.726 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:11.726 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:11.726 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 22, 2024-05-08T22:10:11.727 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 1007, 2024-05-08T22:10:11.727 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-08T22:10:11.727 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 45, 2024-05-08T22:10:11.727 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 1134, 2024-05-08T22:10:11.727 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 16, 2024-05-08T22:10:11.727 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 104, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 1042, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 1761079952, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:11.728 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:11.729 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:11.729 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:11.729 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:11.729 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:11.729 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:11.729 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:11.730 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 8, 2024-05-08T22:10:11.730 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 235, 2024-05-08T22:10:11.730 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 1360, 2024-05-08T22:10:11.730 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:11.730 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:11.730 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:11.731 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 1272, 2024-05-08T22:10:11.731 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 971, 2024-05-08T22:10:11.731 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 29, 2024-05-08T22:10:11.731 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 217 2024-05-08T22:10:11.731 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 158, 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 185, 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 93, 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 288, 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 639, 2024-05-08T22:10:11.732 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-08T22:10:11.733 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:11.733 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:11.733 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 49, 2024-05-08T22:10:11.733 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 34, 2024-05-08T22:10:11.733 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 15, 2024-05-08T22:10:11.733 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:11.734 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 45, 2024-05-08T22:10:11.735 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-08T22:10:11.735 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 3, 2024-05-08T22:10:11.735 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:11.735 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-08T22:10:11.735 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.735 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-08T22:10:11.736 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 3287, 2024-05-08T22:10:11.736 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 3285, 2024-05-08T22:10:11.736 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 3285, 2024-05-08T22:10:11.736 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 2, 2024-05-08T22:10:11.736 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-08T22:10:11.736 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 3, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 73, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 72, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 72, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 1, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 13630154, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 13640682, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-08T22:10:11.737 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1132, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 6.876029278, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006074230 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 48, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 1182, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 1134, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 50, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 182, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 132, 2024-05-08T22:10:11.738 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 43, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 1740, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 1697, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 4, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 1180, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 1176, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 68652, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 207156 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 2613, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 262, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 2389, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 469, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 356, 2024-05-08T22:10:11.739 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1027, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.824083621, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001776128 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 193, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.119358667, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000618438 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.740 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 16, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.028945183, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001809073 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 80, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 3.278825187, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.040985314 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.741 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000210984, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000210984 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.742 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 35, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.122076085, 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.003487888 2024-05-08T22:10:11.743 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 533, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.191083812, 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000358506 2024-05-08T22:10:11.744 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 42, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.101433736, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002415088 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 40, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.163672971, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004091824 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.745 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.746 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001199044, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001199044 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.747 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.748 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 7, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.019472414, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002781773 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-08T22:10:11.749 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 35, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.081059075, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002315973 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-08T22:10:11.750 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 1, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 3, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 3, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 931, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 240, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 240, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:11.751 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:11.752 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:11.753 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 325230, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 142, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 158, 2024-05-08T22:10:11.754 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 213635, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2149, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 3736, 2024-05-08T22:10:11.755 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 11511916, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 37, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 3736, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3736, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 82.824819770, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.022169384 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3736, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 6998 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 3736, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 11, 2024-05-08T22:10:11.756 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 3725, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 540, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 1042, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 11, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 1143, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 233, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 3, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 565, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:11.757 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 2084, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-08T22:10:11.758 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-08T22:10:11.759 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 81, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 3, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 1319, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 58 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:11.760 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 19, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 6, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 34, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:11.761 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 22417, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 5635348, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 22417, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 22417, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 5635348, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.762 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 3736, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 11515350, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3736, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 11515350, 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:11.763 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 3736, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 3736, 2024-05-08T22:10:11.764 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3736, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 3736, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 34, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 3434, 2024-05-08T22:10:11.765 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 34, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 11, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 3434, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x56298e6360a0": { 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 3287, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 9446378, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 3287, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1132, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 9446378, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:11.767 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-08T22:10:12.066 INFO:tasks.check_counter:Daemon mds.b mds.exported=45 2024-05-08T22:10:12.066 INFO:tasks.check_counter:Daemon mds.b mds.imported=16 2024-05-08T22:10:12.067 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=158 2024-05-08T22:10:12.067 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=185 2024-05-08T22:10:12.067 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T22:10:12.067 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T22:10:12.094 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:11 smithi160 ceph-mon[46962]: pgmap v726: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-05-08T22:10:12.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:11 smithi186 ceph-mon[46619]: pgmap v726: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-05-08T22:10:12.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:11 smithi064 ceph-mon[42500]: pgmap v726: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-05-08T22:10:12.291 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:10:13.246 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-08T22:10:13.246 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 312, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 283, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 655058, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 158749, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.038895094, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.013346795, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.018200634, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.006314515, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 282, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.101740804, 2024-05-08T22:10:13.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000360782 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 654224, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 158432 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 927, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1135, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 227892, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 810304, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:13.248 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.080965527, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.025377983, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.017382377, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.030276638, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1133, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.162730507, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000143627 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 522624 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2264, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2264, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3872475, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 703366, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.181067476, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.055523586, 2024-05-08T22:10:13.250 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.050165801, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.060241650, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2263, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.302141257, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000133513 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-08T22:10:13.251 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5413, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.123347084, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000022787 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.252 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:13.253 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 479, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 36, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 443, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 427, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 11, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-05-08T22:10:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2024-05-08T22:10:13.255 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:13.256 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:13.257 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:13.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 1981, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-05-08T22:10:13.259 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 7402470, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 7401332, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 1981 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 208, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 518, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 310, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 24, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 35, 2024-05-08T22:10:13.260 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 11, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 479, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 558, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 79, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 49544, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 207156 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-08T22:10:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.262 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-08T22:10:13.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.264 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-08T22:10:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.267 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-08T22:10:13.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.271 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 1, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2024-05-08T22:10:13.272 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 3696, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:13.273 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:13.274 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 196787, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 250, 2024-05-08T22:10:13.275 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 158, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 831246, 2024-05-08T22:10:13.276 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 7522, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 3186, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 3186, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3186, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 12.754820731, 2024-05-08T22:10:13.277 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004003396 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3186, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 3187 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 3186, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 3186, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 1916, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 1268, 2024-05-08T22:10:13.278 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:13.279 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-08T22:10:13.280 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 81, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 2, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 5, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 6, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 1, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2024-05-08T22:10:13.281 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-05-08T22:10:13.282 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 3499, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 4421578, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 3499, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3499, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 4421578, 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:13.283 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 3186, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 3208171, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3186, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 3208171, 2024-05-08T22:10:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 3186, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 3186, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3186, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 3186, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:13.286 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:13.287 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x55c3516405a0": { 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.289 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:13.289 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-08T22:10:13.615 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-08T22:10:13.615 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-08T22:10:13.615 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-08T22:10:13.615 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-08T22:10:13.615 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T22:10:13.615 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T22:10:13.883 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:10:13.975 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:13 smithi160 ceph-mon[46962]: pgmap v727: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 602 op/s 2024-05-08T22:10:14.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:13 smithi186 ceph-mon[46619]: pgmap v727: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 602 op/s 2024-05-08T22:10:14.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:13 smithi064 ceph-mon[42500]: pgmap v727: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 602 op/s 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4061, 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 3429, 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1519355, 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 4467559, 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:14.802 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.272639561, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.116052201, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.161548317, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.023761383, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3429, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.481498556, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000140419 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5791, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001147593, 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000198 2024-05-08T22:10:14.803 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 3015, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4114, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1466420, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 2214402, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 13, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 8, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.220645311, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.083203776, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.152846464, 2024-05-08T22:10:14.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.015365640, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4112, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.350741490, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000085297 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5010, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000978287, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000195 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 669136, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 163344 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:14.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1870, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1769, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 724424, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1270102, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 12, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 7, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.116222525, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.047014888, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.079242249, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.006898960, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1767, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.199241087, 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000112756 2024-05-08T22:10:14.806 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2659, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000474310, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000178 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 523184 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:14.807 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1227, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.561211153, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000457384 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 887, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.031974362, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000036047 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.808 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 1376, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 1148, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1148, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 6.905238579, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006015016 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 228, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 11, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 251, 2024-05-08T22:10:14.809 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 4, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 222, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 25, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 244, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 74, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 199, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 199, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 21, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 2557, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 1915, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 228, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 34, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:14.810 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 92, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 3734, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 11, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 68, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 20, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 252, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 1043, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 1724666620, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:14.811 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 7, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 826, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 657, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 436, 2024-05-08T22:10:14.812 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 109, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 1045 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 360, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 297, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 159, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 264, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 374, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 12, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 252, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 240, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:14.813 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 11, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-08T22:10:14.814 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 1981, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 1981, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 2, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 7402470, 2024-05-08T22:10:14.815 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 520, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 4.458010853, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.008573097 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 225, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 530, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 305, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 58, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 171, 2024-05-08T22:10:14.816 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 113, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 254, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 570, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 316, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 199, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 602, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 403, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 56904, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 198964 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 1772, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 1376, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 52, 2024-05-08T22:10:14.817 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 86, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 250, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.395320234, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001581280 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.009924874, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004962437 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 443, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 5.943672158, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.013416867 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.819 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000131796, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000131796 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.821 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-08T22:10:14.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 172, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.070299493, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000408717 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 28, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.016982796, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000606528 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 223, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.402863709, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001806563 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:14.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:14.824 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.825 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.826 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 29, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.066043519, 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002277362 2024-05-08T22:10:14.827 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 1, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-08T22:10:14.828 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 126, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 126, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 932, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:14.829 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:14.830 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 292150, 2024-05-08T22:10:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 330, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 704, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 8, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 158, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-08T22:10:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 847424, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 9312, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 1298, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 3307034, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 27, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 1298, 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-08T22:10:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1298, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 9.790862423, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.007543037 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1298, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1501 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 1298, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 107, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 1191, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-08T22:10:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 10, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 107, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 627, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 145, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 252, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 20, 2024-05-08T22:10:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-08T22:10:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 81, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 2, 2024-05-08T22:10:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 251, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 89 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-08T22:10:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 6, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 6, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 240, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 8942, 2024-05-08T22:10:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 2885171, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 8942, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 8942, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2885171, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 1298, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 3331274, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1298, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 3331274, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 1298, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1298, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1298, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1298, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 240, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 24240, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 240, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 107, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 24240, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x55dbff15c0a0": { 2024-05-08T22:10:14.844 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 1981, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 3208166, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 1981, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 520, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 3208166, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:14.845 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.846 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:14.846 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-08T22:10:14.961 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:14 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:15.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:14 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:15.174 INFO:tasks.check_counter:Daemon mds.h mds.exported=11 2024-05-08T22:10:15.174 INFO:tasks.check_counter:Daemon mds.h mds.imported=20 2024-05-08T22:10:15.174 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=360 2024-05-08T22:10:15.174 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=297 2024-05-08T22:10:15.174 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T22:10:15.175 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T22:10:15.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:14 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:15.397 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:10:16.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:15 smithi186 ceph-mon[46619]: pgmap v728: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-05-08T22:10:16.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:15 smithi160 ceph-mon[46962]: pgmap v728: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-05-08T22:10:16.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:15 smithi064 ceph-mon[42500]: pgmap v728: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-05-08T22:10:16.644 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1002, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 948, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1020816, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1614066, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.095143455, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.036023688, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.045467969, 2024-05-08T22:10:16.645 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.015359750, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 948, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.168577080, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000177823 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 693, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000120937, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000174 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 661680, 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 164672 2024-05-08T22:10:16.646 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 3726, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2407, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1535479, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1369065, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 10, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.218284453, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.093523870, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.159095297, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.004727964, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2405, 2024-05-08T22:10:16.647 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.274932214, 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000114316 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5602, 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001479235, 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000264 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T22:10:16.648 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 438944 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1246, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1661, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 487567, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1053625, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 14, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.099731302, 2024-05-08T22:10:16.649 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.036566739, 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.073771687, 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.002772497, 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1659, 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.210987028, 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000127177 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.650 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2382, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000515468, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000216 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-08T22:10:16.651 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 609, 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.097504409, 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000160105 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-05-08T22:10:16.652 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000015528, 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000007764 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 312, 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 267, 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 267, 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.169107695, 2024-05-08T22:10:16.653 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000633362 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 45, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 1, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 11, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-08T22:10:16.654 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 402, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 4, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 398, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 391, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 32, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 373, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 373, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 20, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 836, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 763, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 45, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 23, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 1, 2024-05-08T22:10:16.655 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 7302, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 1, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 32, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 9, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 366, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 1043, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 1724666620, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-08T22:10:16.656 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 3, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 108, 2024-05-08T22:10:16.657 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 224, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 177, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 8, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 7 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 130, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 558, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 290, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 512, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 135, 2024-05-08T22:10:16.658 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 1, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-08T22:10:16.659 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 758, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 758, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-08T22:10:16.660 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 1, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 1, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 5367797, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 267, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 3.263964107, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.012224584 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-08T22:10:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 373, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 405, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 32, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 47, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 286, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 239, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 370, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 408, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 38, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 341, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 661, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 320, 2024-05-08T22:10:16.662 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 53912, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 198964 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 314, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 312, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 18, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.663 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.664 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 232, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.142298983, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000613357 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.665 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.666 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-08T22:10:16.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-08T22:10:16.668 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 26, 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.018022780, 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000693183 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 9, 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.008785932, 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000976214 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-08T22:10:16.669 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.670 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.671 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.673 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-08T22:10:16.674 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 1, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 48, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 48, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 926, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:16.675 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:16.676 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:16.677 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 303873, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 501, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:16.678 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 158, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 1402665, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 13836, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:16.679 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 377, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 1212517, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 26, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 377, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 377, 2024-05-08T22:10:16.680 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 4.268939986, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.011323448 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 377, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 423 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 377, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 1, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 376, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 0, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 10, 2024-05-08T22:10:16.681 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 267, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 59, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 6, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 20, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:16.682 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-08T22:10:16.683 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 81, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 2, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 9, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-08T22:10:16.684 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 3, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 51, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 8 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.685 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 5970, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 2486375, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:16.686 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 5970, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 5970, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2486375, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:16.687 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 377, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1212501, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 377, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1212501, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:16.688 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 377, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 377, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 377, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 377, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.689 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:16.690 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x562ed16680a0": { 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 758, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1173493, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 758, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-08T22:10:16.691 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 267, 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1173493, 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-08T22:10:16.692 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-08T22:10:16.999 INFO:tasks.check_counter:Daemon mds.k mds.exported=1 2024-05-08T22:10:16.999 INFO:tasks.check_counter:Daemon mds.k mds.imported=9 2024-05-08T22:10:16.999 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=130 2024-05-08T22:10:16.999 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=558 2024-05-08T22:10:16.999 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T22:10:17.000 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T22:10:17.044 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:16 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:17.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:16 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:17.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:16 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:17.251 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:10:18.032 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:17 smithi186 ceph-mon[46619]: pgmap v729: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-08T22:10:18.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:17 smithi064 ceph-mon[42500]: pgmap v729: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1104, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1106, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 4557927, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 343354, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.112674514, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.030744947, 2024-05-08T22:10:18.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.032327634, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.038385471, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1104, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.288743843, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000261543 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.216 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1908, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1874, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2264365, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 653301, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.157786171, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.045952319, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.048521115, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.054922474, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1874, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.443189852, 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000236494 2024-05-08T22:10:18.217 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 675472, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 159056 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 609, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 817, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 4349985, 2024-05-08T22:10:18.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 712970, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.058227194, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.022691112, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.014997573, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.014796134, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 815, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.109148866, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000133924 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.219 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 524192 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 5527, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.163356207, 2024-05-08T22:10:18.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000029556 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.221 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 66, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 37, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 29, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-05-08T22:10:18.222 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 12, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-05-08T22:10:18.223 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 1042, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 1761079952, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:18.224 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-05-08T22:10:18.225 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:18.226 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 3303, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-08T22:10:18.227 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 13630154, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 13771279, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 13764525, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 3303 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.228 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 193, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 1345, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 1152, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 22, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 80, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 58, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 244, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 1396, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 1152, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 49520, 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 207156 2024-05-08T22:10:18.229 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.230 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-08T22:10:18.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-08T22:10:18.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-08T22:10:18.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.236 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.238 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.239 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 3702, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.241 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:18.242 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:18.243 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 190228, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 53, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-08T22:10:18.244 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 158, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 103806, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 489, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-08T22:10:18.245 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 3300, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 26, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 4, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 3300, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3300, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 10.003827047, 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.003031462 2024-05-08T22:10:18.246 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3300, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 3301 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 3300, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 3300, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1982, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1315, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-05-08T22:10:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 2, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:18.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-08T22:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 81, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 2, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 5, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 5, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-05-08T22:10:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 3617, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 10827629, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 3617, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3617, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 10827629, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 3300, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 9576985, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3300, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 9576985, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 3300, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 3300, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3300, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 3300, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-08T22:10:18.256 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55c325e3e640": { 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:18.258 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-08T22:10:18.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:17 smithi160 ceph-mon[46962]: pgmap v729: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-08T22:10:18.594 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T22:10:18.594 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T22:10:18.594 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-08T22:10:18.594 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-08T22:10:18.595 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T22:10:18.595 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T22:10:18.816 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:10:18.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:18 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:18.891 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:18 smithi186 ceph-mon[46619]: pgmap v730: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T22:10:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:18 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:19.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:18 smithi064 ceph-mon[42500]: pgmap v730: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T22:10:19.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:18 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:19.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:18 smithi160 ceph-mon[46962]: pgmap v730: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T22:10:19.824 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-08T22:10:19.824 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:19.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 3169, 2024-05-08T22:10:19.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 3171, 2024-05-08T22:10:19.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 7233754, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 985566, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.232879987, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.063865751, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.063502147, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.088377722, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3169, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.538694596, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000169988 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.825 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 318, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 285, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 679196, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 159724, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-05-08T22:10:19.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.038751613, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.013698228, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.018272081, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006731743, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 284, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.073169287, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000257638 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 678640, 2024-05-08T22:10:19.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 159040 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 7, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 214, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3580, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 525127, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.011926285, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.007839724, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.002274679, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000031992, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 213, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.080230575, 2024-05-08T22:10:19.828 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000376669 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 524816 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-08T22:10:19.829 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 5232, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.119513305, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000022842 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.830 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-05-08T22:10:19.831 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 942, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 942, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 942, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-05-08T22:10:19.832 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-08T22:10:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:19.834 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:19.835 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-08T22:10:19.836 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 3606, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 5, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-08T22:10:19.837 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 10541903, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 10540647, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 3606 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 80, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 190, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 110, 2024-05-08T22:10:19.838 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 15, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 27, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 12, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 942, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 1053, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 111, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 51216, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 207156 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-05-08T22:10:19.839 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.840 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.841 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:19.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.843 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.844 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.846 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-05-08T22:10:19.850 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 3706, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:19.851 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:19.852 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:19.853 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 184587, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 115, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 158, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:19.854 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 884583, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 27269, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 3172, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-05-08T22:10:19.855 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 3172, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3172, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 15.084739614, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.004755592 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3172, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 3173 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 3172, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 3172, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-05-08T22:10:19.856 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1924, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1246, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:19.857 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-08T22:10:19.858 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-08T22:10:19.859 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 81, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 2, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 4, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 5, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 1, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:19.860 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 3490, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 7583261, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 3490, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3490, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 7583261, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.862 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 3172, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 6347604, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3172, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 6347604, 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:19.863 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 3172, 2024-05-08T22:10:19.864 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 3172, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3172, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 3172, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:19.865 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x560def0225a0": { 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:19.866 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:19.867 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-08T22:10:20.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:19 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:20.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:19 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:20.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:19 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:20.688 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T22:10:20.689 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T22:10:20.689 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-08T22:10:20.689 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-08T22:10:20.689 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T22:10:20.689 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T22:10:20.874 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:20 smithi186 ceph-mon[46619]: pgmap v731: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-05-08T22:10:20.917 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:10:21.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:20 smithi064 ceph-mon[42500]: pgmap v731: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-05-08T22:10:21.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:20 smithi160 ceph-mon[46962]: pgmap v731: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-05-08T22:10:21.864 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-08T22:10:21.864 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:21.864 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 8888, 2024-05-08T22:10:21.864 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 10160, 2024-05-08T22:10:21.864 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3147250, 2024-05-08T22:10:21.864 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 9830310, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 9, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.563290006, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.217981523, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.305104016, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.067474910, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 10161, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.912766914, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000089830 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 13190, 2024-05-08T22:10:21.865 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.002364051, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000179 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 694144, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 162960 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 22, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 234, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 5700, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 533408, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 13, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 7, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.015363808, 2024-05-08T22:10:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008116389, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.004928079, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000380560, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 228, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.058830284, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000258027 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 11, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000001553, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000141 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1904, 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 527200 2024-05-08T22:10:21.867 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 9204, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 10257, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2650887, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 5535540, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 13, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.476346229, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.198049249, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.360770636, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.007512307, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 10258, 2024-05-08T22:10:21.868 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.841788821, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000082061 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 16325, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.002751347, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000168 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.869 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4700, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2.091001630, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000444893 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 37, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000982138, 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000026544 2024-05-08T22:10:21.870 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 3941, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 3883, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3883, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 9.964810548, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002566265 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 58, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-08T22:10:21.871 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 963, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 7, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 840, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 116, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 949, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 108, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 71, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 71, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 10, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 11892, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 9307, 2024-05-08T22:10:21.872 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 50, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 30, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 1022, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 8039, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 7, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 103, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 8, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 120, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:21.873 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 1043, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 1724666620, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 40, 2024-05-08T22:10:21.874 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 911, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 1812, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 2253, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 836, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 85, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 2463 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 215, 2024-05-08T22:10:21.875 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 613, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 320, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 563, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 316, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 603, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 4, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 599, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:21.876 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 8, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 3608, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 3608, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-08T22:10:21.877 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 5, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 5, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 10543733, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2351, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 10.796336895, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.004592231 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.878 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 106, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 211, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 105, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 41, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 319, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 278, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 963, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 1075, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 112, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 71, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 520, 2024-05-08T22:10:21.879 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 449, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 54580, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 198964 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 8132, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 23, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 3941, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 1176, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 279, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 52, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.039828674, 2024-05-08T22:10:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000765936 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 41, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.087630436, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002137327 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-08T22:10:21.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 598, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.446271455, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002418514 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1848, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2.422031943, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001310623 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:21.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.884 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 115, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.139033779, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001208989 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.885 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 27, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.043311442, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001604127 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 601, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 4.602477796, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.007658032 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.887 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:21.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-08T22:10:21.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 601, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.184225023, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001970424 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.890 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2024-05-08T22:10:21.891 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 313, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 313, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 936, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:21.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:21.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:21.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 289578, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 192, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:21.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 158, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 1081318, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 35461, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-08T22:10:21.896 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2629, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 6437019, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 27, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2629, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2629, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 15.651729352, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.005953491 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2629, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2850 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2629, 2024-05-08T22:10:21.897 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 4, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 2625, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 4, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 2355, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 71, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 17, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-08T22:10:21.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:21.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-08T22:10:21.900 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 81, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 2, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 10, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 12, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 2, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 226, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 147 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-05-08T22:10:21.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 1, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 1, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 4, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.902 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 18110, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 4153282, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 18110, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 18110, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 4153282, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.903 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2629, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 6437423, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2629, 2024-05-08T22:10:21.904 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 6437423, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2629, 2024-05-08T22:10:21.905 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2629, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2629, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2629, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:21.906 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 4, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 404, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 4, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 4, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 404, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5605617960a0": { 2024-05-08T22:10:21.907 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 1830, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 3608, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 6349429, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 3608, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2351, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 6347599, 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:21.909 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:21.909 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:21.909 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.909 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:21.909 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-08T22:10:22.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:21 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:22.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:21 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:22.227 INFO:tasks.check_counter:Daemon mds.i mds.exported=7 2024-05-08T22:10:22.227 INFO:tasks.check_counter:Daemon mds.i mds.imported=8 2024-05-08T22:10:22.227 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=215 2024-05-08T22:10:22.227 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=613 2024-05-08T22:10:22.227 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T22:10:22.227 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 71e641de-0d84-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T22:10:22.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:21 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:22.450 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:10:23.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:22 smithi186 ceph-mon[46619]: pgmap v732: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-05-08T22:10:23.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:22 smithi064 ceph-mon[42500]: pgmap v732: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-05-08T22:10:23.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:22 smithi160 ceph-mon[46962]: pgmap v732: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2696, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 3300, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 845458, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 3409503, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 15, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.193475416, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.071565985, 2024-05-08T22:10:23.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.124038197, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.017393539, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3299, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.539793935, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000163623 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4305, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000904882, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000210 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:10:23.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1735, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2706, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1289645, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 1090418, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 9, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.142247955, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.059780312, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.089253963, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.007103607, 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:23.458 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2705, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.519351470, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000191996 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3449, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000654325, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000189 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 692496, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 165968 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:10:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 3082, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2888, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1393129, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 1604195, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 11, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.189716784, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.076091819, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.134161958, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005410870, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2886, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.505758322, 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000175245 2024-05-08T22:10:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 5402, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001150652, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000213 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 528032 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 992, 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.229719814, 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000231572 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 23, 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000419957, 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000018259 2024-05-08T22:10:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 620, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 355, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 355, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.218006725, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000614103 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 265, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 1, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-08T22:10:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 351, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 2, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 8, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 341, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 347, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 138, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 318, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 318, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 27, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 1083, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 744, 2024-05-08T22:10:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 256, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 21, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 9580, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 6, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 145, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 11, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 469, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 1043, 2024-05-08T22:10:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 1724666620, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 12, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 60, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 138, 2024-05-08T22:10:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 1, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 259, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 134, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 9, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 33 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 69, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 302, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 161, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 292, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 455, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-08T22:10:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 2, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 2, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 6, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:10:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 850, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 850, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 1, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 1, 2024-05-08T22:10:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 6024571, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 499, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 6.517037521, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.013060195 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-05-08T22:10:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 345, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 476, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 131, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 61, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 184, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 123, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 351, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 499, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 148, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 318, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 666, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 348, 2024-05-08T22:10:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 54892, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 198964 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 638, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 61, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 620, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 200, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 122, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.007128083, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002376027 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.472 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 6, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.028368207, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.004728034 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.473 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 291, 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.109309707, 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000375634 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.474 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.475 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.476 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 31, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.025814067, 2024-05-08T22:10:23.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000832711 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 18, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.023638429, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001313246 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.004455779, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001485259 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.478 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-08T22:10:23.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.006727083, 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.006727083 2024-05-08T22:10:23.480 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.481 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-05-08T22:10:23.482 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012565370, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.006282685 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2024-05-08T22:10:23.483 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 1, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 72, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 72, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 946, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:10:23.484 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:10:23.485 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-08T22:10:23.486 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 300791, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 431, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 176, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 2, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-08T22:10:23.487 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7528, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 158, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 1233468, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 11597, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.488 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 634, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 1866632, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 33, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 634, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 634, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 7.888864997, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.012443004 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 634, 2024-05-08T22:10:23.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 689 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 634, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 3, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 631, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 2, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 501, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 62, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-08T22:10:23.490 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 14, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-08T22:10:23.491 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 2, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-08T22:10:23.492 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 81, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 2, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 10, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 12, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 2, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 61, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-05-08T22:10:23.493 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 15 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 1, 2024-05-08T22:10:23.494 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 1, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 2, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 7509, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2831660, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 7509, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:23.495 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 7509, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2831660, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:23.496 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 634, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1866818, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 634, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1866818, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-08T22:10:23.497 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 634, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 634, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 634, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 634, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.498 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 202, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 202, 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:23.499 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55a34e7ae0a0": { 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 574, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 850, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1830267, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 850, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-08T22:10:23.500 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 499, 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1829693, 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-08T22:10:23.501 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-08T22:10:23.869 INFO:tasks.check_counter:Daemon mds.l mds.exported=6 2024-05-08T22:10:23.869 INFO:tasks.check_counter:Daemon mds.l mds.imported=11 2024-05-08T22:10:23.870 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=69 2024-05-08T22:10:23.870 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=302 2024-05-08T22:10:23.870 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T22:10:23.881 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T22:10:23.881 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T22:10:23.881 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:10:23.907 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-08T22:10:23.907 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:10:23.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:10:24.141 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:23 smithi186 ceph-mon[46619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 -- 172.21.15.64:0/860320856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c100f80 msgr2=0x7f5f8c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 --2- 172.21.15.64:0/860320856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c100f80 0x7f5f8c1013e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0099c0 tx=0x7f5f7c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 -- 172.21.15.64:0/860320856 shutdown_connections 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 --2- 172.21.15.64:0/860320856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f8c0ffd90 0x7f5f8c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 --2- 172.21.15.64:0/860320856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f8c101920 0x7f5f8c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 --2- 172.21.15.64:0/860320856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c100f80 0x7f5f8c1013e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 -- 172.21.15.64:0/860320856 >> 172.21.15.64:0/860320856 conn(0x7f5f8c0fb300 msgr2=0x7f5f8c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 -- 172.21.15.64:0/860320856 shutdown_connections 2024-05-08T22:10:24.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.152+0000 7f5f90d46700 1 -- 172.21.15.64:0/860320856 wait complete. 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 Processor -- start 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 -- start start 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f8c0ffd90 0x7f5f8c103cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f8c100f80 0x7f5f8c102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 0x7f5f8c1028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c103040 con 0x7f5f8c101920 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1031b0 con 0x7f5f8c0ffd90 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f90d46700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1a0e10 con 0x7f5f8c100f80 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f8ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 0x7f5f8c1028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f8ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 0x7f5f8c1028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58498/0 (socket says 172.21.15.64:58498) 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f8ad9d700 1 -- 172.21.15.64:0/2314506639 learned_addr learned my addr 172.21.15.64:0/2314506639 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f89d9b700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f8c100f80 0x7f5f8c102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.153+0000 7f5f8a59c700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f8c0ffd90 0x7f5f8c103cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f8ad9d700 1 -- 172.21.15.64:0/2314506639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f8c100f80 msgr2=0x7f5f8c102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f8ad9d700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f8c100f80 0x7f5f8c102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f8ad9d700 1 -- 172.21.15.64:0/2314506639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f8c0ffd90 msgr2=0x7f5f8c103cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f8ad9d700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f8c0ffd90 0x7f5f8c103cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f8ad9d700 1 -- 172.21.15.64:0/2314506639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0095f0 con 0x7f5f8c101920 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f8ad9d700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 0x7f5f8c1028a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ef30 tx=0x7f5f8400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84018070 con 0x7f5f8c101920 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c1a1040 con 0x7f5f8c101920 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f8400f040 con 0x7f5f8c101920 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400ce00 con 0x7f5f8c101920 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.154+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c1a1500 con 0x7f5f8c101920 2024-05-08T22:10:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.155+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c067380 con 0x7f5f8c101920 2024-05-08T22:10:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.156+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5f84003a00 con 0x7f5f8c101920 2024-05-08T22:10:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.157+0000 7f5f7b7fe700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5f740a1120 0x7f5f740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.157+0000 7f5f8a59c700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5f740a1120 0x7f5f740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.157+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f840ccf20 con 0x7f5f8c101920 2024-05-08T22:10:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.158+0000 7f5f8a59c700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5f740a1120 0x7f5f740a35e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.161+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5f84096750 con 0x7f5f8c101920 2024-05-08T22:10:24.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:23 smithi064 ceph-mon[42500]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:24.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:23 smithi160 ceph-mon[46962]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:10:24.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.370+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f8c10b5a0 con 0x7f5f8c101920 2024-05-08T22:10:24.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.372+0000 7f5f7b7fe700 1 -- 172.21.15.64:0/2314506639 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22495 (secure 0 0 0) 0x7f5f84099d70 con 0x7f5f8c101920 2024-05-08T22:10:24.372 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:24.373 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":81,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:54:21.242448+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5f740a1120 msgr2=0x7f5f740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5f740a1120 0x7f5f740a35e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f800058e0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 msgr2=0x7f5f8c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 0x7f5f8c1028a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ef30 tx=0x7f5f8400c450 comp rx=0 tx=0).stop 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 shutdown_connections 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f5f740a1120 0x7f5f740a35e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f8c0ffd90 0x7f5f8c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c101920 0x7f5f8c1028a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 --2- 172.21.15.64:0/2314506639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f8c100f80 0x7f5f8c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 >> 172.21.15.64:0/2314506639 conn(0x7f5f8c0fb300 msgr2=0x7f5f8c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 shutdown_connections 2024-05-08T22:10:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.374+0000 7f5f90d46700 1 -- 172.21.15.64:0/2314506639 wait complete. 2024-05-08T22:10:24.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:50:08.557347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:53:21.578075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:53:22.933075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:10:24.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:10:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 -- 172.21.15.64:0/2688819213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8700ffe60 msgr2=0x7fb870100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 --2- 172.21.15.64:0/2688819213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8700ffe60 0x7fb870100280 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb864009990 tx=0x7fb864009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 -- 172.21.15.64:0/2688819213 shutdown_connections 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 --2- 172.21.15.64:0/2688819213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb870101970 0x7fb870108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 --2- 172.21.15.64:0/2688819213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8700ffe60 0x7fb870100280 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 --2- 172.21.15.64:0/2688819213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb870100fb0 0x7fb870101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 -- 172.21.15.64:0/2688819213 >> 172.21.15.64:0/2688819213 conn(0x7fb8700fb3f0 msgr2=0x7fb8700fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 -- 172.21.15.64:0/2688819213 shutdown_connections 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.517+0000 7fb876cf2700 1 -- 172.21.15.64:0/2688819213 wait complete. 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 Processor -- start 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 -- start start 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8700ffe60 0x7fb870068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 0x7fb87019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb870101970 0x7fb8701a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701a0b40 con 0x7fb870100fb0 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701a0cb0 con 0x7fb870101970 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb876cf2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701a0e70 con 0x7fb8700ffe60 2024-05-08T22:10:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 0x7fb87019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 0x7fb87019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58512/0 (socket says 172.21.15.64:58512) 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb86ffff700 1 -- 172.21.15.64:0/1466947433 learned_addr learned my addr 172.21.15.64:0/1466947433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb874a8e700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8700ffe60 0x7fb870068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb87528f700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb870101970 0x7fb8701a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb86ffff700 1 -- 172.21.15.64:0/1466947433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8700ffe60 msgr2=0x7fb870068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.518+0000 7fb86ffff700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8700ffe60 0x7fb870068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86ffff700 1 -- 172.21.15.64:0/1466947433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb870101970 msgr2=0x7fb8701a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86ffff700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb870101970 0x7fb8701a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86ffff700 1 -- 172.21.15.64:0/1466947433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640095f0 con 0x7fb870100fb0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86ffff700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 0x7fb87019df90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00c9b0 tx=0x7fb85c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c007b00 con 0x7fb870100fb0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb85c00ce50 con 0x7fb870100fb0 2024-05-08T22:10:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8701a1150 con 0x7fb870100fb0 2024-05-08T22:10:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c018660 con 0x7fb870100fb0 2024-05-08T22:10:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.519+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8701a16b0 con 0x7fb870100fb0 2024-05-08T22:10:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.520+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb870107280 con 0x7fb870100fb0 2024-05-08T22:10:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.521+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb85c00f450 con 0x7fb870100fb0 2024-05-08T22:10:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.522+0000 7fb86dffb700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8600a11a0 0x7fb8600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.522+0000 7fb874a8e700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8600a11a0 0x7fb8600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.522+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb85c0cc810 con 0x7fb870100fb0 2024-05-08T22:10:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.522+0000 7fb874a8e700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8600a11a0 0x7fb8600a3660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b2e0 tx=0x7fb86400b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.526+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb85c095f50 con 0x7fb870100fb0 2024-05-08T22:10:24.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.735+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb870067cc0 con 0x7fb870100fb0 2024-05-08T22:10:24.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.735+0000 7fb86dffb700 1 -- 172.21.15.64:0/1466947433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb85c007e60 con 0x7fb870100fb0 2024-05-08T22:10:24.736 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-08T22:10:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8600a11a0 msgr2=0x7fb8600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8600a11a0 0x7fb8600a3660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b2e0 tx=0x7fb86400b6f0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 msgr2=0x7fb87019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 0x7fb87019df90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00c9b0 tx=0x7fb85c00ccc0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 shutdown_connections 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fb8600a11a0 0x7fb8600a3660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.737+0000 7fb876cf2700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb870101970 0x7fb8701a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.738+0000 7fb876cf2700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8700ffe60 0x7fb870068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.738+0000 7fb876cf2700 1 --2- 172.21.15.64:0/1466947433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb870100fb0 0x7fb87019df90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.738+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 >> 172.21.15.64:0/1466947433 conn(0x7fb8700fb3f0 msgr2=0x7fb8701043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.738+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 shutdown_connections 2024-05-08T22:10:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.738+0000 7fb876cf2700 1 -- 172.21.15.64:0/1466947433 wait complete. 2024-05-08T22:10:24.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:10:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 -- 172.21.15.64:0/2110807917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 msgr2=0x7fa0e8101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/2110807917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8101430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009990 tx=0x7fa0e4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 -- 172.21.15.64:0/2110807917 shutdown_connections 2024-05-08T22:10:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/2110807917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e8101970 0x7fa0e8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/2110807917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e80ffe60 0x7fa0e8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/2110807917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8101430 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 -- 172.21.15.64:0/2110807917 >> 172.21.15.64:0/2110807917 conn(0x7fa0e80fb3f0 msgr2=0x7fa0e80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 -- 172.21.15.64:0/2110807917 shutdown_connections 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.879+0000 7fa0ef74a700 1 -- 172.21.15.64:0/2110807917 wait complete. 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 Processor -- start 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 -- start start 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e80ffe60 0x7fa0e8103d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e8101970 0x7fa0e8102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8103100 con 0x7fa0e8100fb0 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8103270 con 0x7fa0e80ffe60 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ef74a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e81a0f30 con 0x7fa0e8101970 2024-05-08T22:10:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ecce5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ecce5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8102420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58532/0 (socket says 172.21.15.64:58532) 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.880+0000 7fa0ecce5700 1 -- 172.21.15.64:0/3116475434 learned_addr learned my addr 172.21.15.64:0/3116475434 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ed4e6700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e80ffe60 0x7fa0e8103d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0edce7700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e8101970 0x7fa0e8102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ecce5700 1 -- 172.21.15.64:0/3116475434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e8101970 msgr2=0x7fa0e8102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ecce5700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e8101970 0x7fa0e8102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ecce5700 1 -- 172.21.15.64:0/3116475434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e80ffe60 msgr2=0x7fa0e8103d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ecce5700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e80ffe60 0x7fa0e8103d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ecce5700 1 -- 172.21.15.64:0/3116475434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40095f0 con 0x7fa0e8100fb0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ecce5700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8102420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4006010 tx=0x7fa0e4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e401d070 con 0x7fa0e8100fb0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0e4004ca0 con 0x7fa0e8100fb0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e81a1160 con 0x7fa0e8100fb0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4005290 con 0x7fa0e8100fb0 2024-05-08T22:10:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.881+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e81a1520 con 0x7fa0e8100fb0 2024-05-08T22:10:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.882+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e8067cf0 con 0x7fa0e8100fb0 2024-05-08T22:10:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.886+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa0e4003c50 con 0x7fa0e8100fb0 2024-05-08T22:10:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.887+0000 7fa0da7fc700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0d40a1120 0x7fa0d40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.887+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0e40cdd00 con 0x7fa0e8100fb0 2024-05-08T22:10:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.887+0000 7fa0ed4e6700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0d40a1120 0x7fa0d40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.887+0000 7fa0ed4e6700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0d40a1120 0x7fa0d40a35e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00bbd0 tx=0x7fa0dc00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:24.890+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0e40974d0 con 0x7fa0e8100fb0 2024-05-08T22:10:25.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.101+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0e810b7f0 con 0x7fa0e8100fb0 2024-05-08T22:10:25.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.101+0000 7fa0da7fc700 1 -- 172.21.15.64:0/3116475434 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0e810b7f0 con 0x7fa0e8100fb0 2024-05-08T22:10:25.102 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-08T22:10:25.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0d40a1120 msgr2=0x7fa0d40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0d40a1120 0x7fa0d40a35e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00bbd0 tx=0x7fa0dc00b5b0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 msgr2=0x7fa0e8102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8102420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4006010 tx=0x7fa0e4009ee0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 shutdown_connections 2024-05-08T22:10:25.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fa0d40a1120 0x7fa0d40a35e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0e80ffe60 0x7fa0e8103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0e8101970 0x7fa0e8102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 --2- 172.21.15.64:0/3116475434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e8100fb0 0x7fa0e8102420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 >> 172.21.15.64:0/3116475434 conn(0x7fa0e80fb3f0 msgr2=0x7fa0e8068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 shutdown_connections 2024-05-08T22:10:25.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.103+0000 7fa0ef74a700 1 -- 172.21.15.64:0/3116475434 wait complete. 2024-05-08T22:10:25.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2314506639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:25.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:24 smithi064 ceph-mon[42500]: pgmap v733: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-05-08T22:10:25.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:24 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1466947433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:10:25.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 -- 172.21.15.64:0/4007283166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d81019e0 msgr2=0x7f09d8103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/4007283166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d81019e0 0x7f09d8103dd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009990 tx=0x7f09c0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 -- 172.21.15.64:0/4007283166 shutdown_connections 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/4007283166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d8106c40 0x7f09d81090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/4007283166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d81019e0 0x7f09d8103dd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/4007283166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8104310 0x7f09d8106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 -- 172.21.15.64:0/4007283166 >> 172.21.15.64:0/4007283166 conn(0x7f09d80fb3f0 msgr2=0x7f09d80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 -- 172.21.15.64:0/4007283166 shutdown_connections 2024-05-08T22:10:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.246+0000 7f09dcdf3700 1 -- 172.21.15.64:0/4007283166 wait complete. 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 Processor -- start 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 -- start start 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d81019e0 0x7f09d8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 0x7f09d81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8106c40 0x7f09d819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d819e9b0 con 0x7f09d8104310 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d819eaf0 con 0x7f09d8106c40 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09dcdf3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d819ec80 con 0x7f09d81019e0 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 0x7f09d81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 0x7f09d81973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58554/0 (socket says 172.21.15.64:58554) 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 -- 172.21.15.64:0/3470276078 learned_addr learned my addr 172.21.15.64:0/3470276078 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d659c700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d81019e0 0x7f09d8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d6d9d700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8106c40 0x7f09d819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 -- 172.21.15.64:0/3470276078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d81019e0 msgr2=0x7f09d8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d81019e0 0x7f09d8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 -- 172.21.15.64:0/3470276078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8106c40 msgr2=0x7f09d819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8106c40 0x7f09d819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d5d9b700 1 -- 172.21.15.64:0/3470276078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00095f0 con 0x7f09d8104310 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d659c700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d81019e0 0x7f09d8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.247+0000 7f09d6d9d700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8106c40 0x7f09d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.248+0000 7f09d5d9b700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 0x7f09d81973c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c9b0 tx=0x7f09c800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.248+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8007b00 con 0x7f09d8104310 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.248+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c8007c60 con 0x7f09d8104310 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.248+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d819ef60 con 0x7f09d8104310 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.248+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8018660 con 0x7f09d8104310 2024-05-08T22:10:25.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.248+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d819f3e0 con 0x7f09d8104310 2024-05-08T22:10:25.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.249+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d80fcf70 con 0x7f09d8104310 2024-05-08T22:10:25.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.251+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f09c8018880 con 0x7f09d8104310 2024-05-08T22:10:25.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.251+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f09c40a1120 0x7f09c40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.251+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09c80ccbe0 con 0x7f09d8104310 2024-05-08T22:10:25.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.251+0000 7f09d659c700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f09c40a1120 0x7f09c40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.252+0000 7f09d659c700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f09c40a1120 0x7f09c40a35e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b5c0 tx=0x7f09c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.255+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09c8096410 con 0x7f09d8104310 2024-05-08T22:10:25.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:24 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2314506639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:25.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:24 smithi160 ceph-mon[46962]: pgmap v733: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-05-08T22:10:25.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:24 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1466947433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:10:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:24 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2314506639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:24 smithi186 ceph-mon[46619]: pgmap v733: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-05-08T22:10:25.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:24 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1466947433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:10:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.464+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09d810c7c0 con 0x7f09d8104310 2024-05-08T22:10:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.464+0000 7f09cf7fe700 1 -- 172.21.15.64:0/3470276078 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09c8099a30 con 0x7f09d8104310 2024-05-08T22:10:25.465 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-08T22:10:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f09c40a1120 msgr2=0x7f09c40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f09c40a1120 0x7f09c40a35e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b5c0 tx=0x7f09c0005fb0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 msgr2=0x7f09d81973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 0x7f09d81973c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c9b0 tx=0x7f09c800cd70 comp rx=0 tx=0).stop 2024-05-08T22:10:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 shutdown_connections 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f09c40a1120 0x7f09c40a35e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d8106c40 0x7f09d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d81019e0 0x7f09d8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 --2- 172.21.15.64:0/3470276078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d8104310 0x7f09d81973c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.466+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 >> 172.21.15.64:0/3470276078 conn(0x7f09d80fb3f0 msgr2=0x7f09d8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.467+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 shutdown_connections 2024-05-08T22:10:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.467+0000 7f09dcdf3700 1 -- 172.21.15.64:0/3470276078 wait complete. 2024-05-08T22:10:25.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:10:25.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 -- 172.21.15.64:0/3614120956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 msgr2=0x7fcd5c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 --2- 172.21.15.64:0/3614120956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c1093d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400b0a0 tx=0x7fcd5400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 -- 172.21.15.64:0/3614120956 shutdown_connections 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 --2- 172.21.15.64:0/3614120956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd5c100f20 0x7fcd5c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 --2- 172.21.15.64:0/3614120956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c1093d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 --2- 172.21.15.64:0/3614120956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd5c1020f0 0x7fcd5c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 -- 172.21.15.64:0/3614120956 >> 172.21.15.64:0/3614120956 conn(0x7fcd5c0fc4e0 msgr2=0x7fcd5c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 -- 172.21.15.64:0/3614120956 shutdown_connections 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.607+0000 7fcd62841700 1 -- 172.21.15.64:0/3614120956 wait complete. 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 Processor -- start 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 -- start start 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd5c100f20 0x7fcd5c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd5c1020f0 0x7fcd5c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c074070 con 0x7fcd5c102ab0 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0741e0 con 0x7fcd5c1020f0 2024-05-08T22:10:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd62841700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1a1020 con 0x7fcd5c100f20 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd60dde700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd60dde700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58566/0 (socket says 172.21.15.64:58566) 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd60dde700 1 -- 172.21.15.64:0/2359103060 learned_addr learned my addr 172.21.15.64:0/2359103060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd5bfff700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd5c100f20 0x7fcd5c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.608+0000 7fcd5b7fe700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd5c1020f0 0x7fcd5c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd60dde700 1 -- 172.21.15.64:0/2359103060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd5c100f20 msgr2=0x7fcd5c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd60dde700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd5c100f20 0x7fcd5c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd60dde700 1 -- 172.21.15.64:0/2359103060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd5c1020f0 msgr2=0x7fcd5c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd60dde700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd5c1020f0 0x7fcd5c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd60dde700 1 -- 172.21.15.64:0/2359103060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd54009d00 con 0x7fcd5c102ab0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd60dde700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c0738d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400b780 tx=0x7fcd5400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400e040 con 0x7fcd5c102ab0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd54009530 con 0x7fcd5c102ab0 2024-05-08T22:10:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c1a1250 con 0x7fcd5c102ab0 2024-05-08T22:10:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54005280 con 0x7fcd5c102ab0 2024-05-08T22:10:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.609+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c1a1670 con 0x7fcd5c102ab0 2024-05-08T22:10:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.610+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c108340 con 0x7fcd5c102ab0 2024-05-08T22:10:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.611+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcd540053e0 con 0x7fcd5c102ab0 2024-05-08T22:10:25.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.612+0000 7fcd597fa700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcd440a1120 0x7fcd440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.612+0000 7fcd5bfff700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcd440a1120 0x7fcd440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.612+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd540cdf40 con 0x7fcd5c102ab0 2024-05-08T22:10:25.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.612+0000 7fcd5bfff700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcd440a1120 0x7fcd440a35e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c100c40 tx=0x7fcd4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.616+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcd54097770 con 0x7fcd5c102ab0 2024-05-08T22:10:25.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.839+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd5c10c8f0 con 0x7fcd5c102ab0 2024-05-08T22:10:25.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.840+0000 7fcd597fa700 1 -- 172.21.15.64:0/2359103060 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) v1 ==== 81+0+590 (secure 0 0 0) 0x7fcd5409ad90 con 0x7fcd5c102ab0 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1096250282 2024-05-09T21:48:20.034470+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/865555054 2024-05-09T21:48:20.034470+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/865555054 2024-05-09T21:48:20.034470+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2750019664 2024-05-09T21:48:20.034470+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3310656394 2024-05-09T21:47:59.744672+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/1406677494 2024-05-09T21:47:59.744672+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1642224217 2024-05-09T21:47:59.744672+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3843761611 2024-05-09T21:47:59.744672+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/1406677494 2024-05-09T21:47:59.744672+0000 2024-05-08T22:10:25.841 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3369547289 2024-05-09T21:48:20.034470+0000 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcd440a1120 msgr2=0x7fcd440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcd440a1120 0x7fcd440a35e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c100c40 tx=0x7fcd4c009260 comp rx=0 tx=0).stop 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 msgr2=0x7fcd5c0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c0738d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400b780 tx=0x7fcd5400bc80 comp rx=0 tx=0).stop 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 shutdown_connections 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fcd440a1120 0x7fcd440a35e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd5c102ab0 0x7fcd5c0738d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd5c100f20 0x7fcd5c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 --2- 172.21.15.64:0/2359103060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd5c1020f0 0x7fcd5c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 >> 172.21.15.64:0/2359103060 conn(0x7fcd5c0fc4e0 msgr2=0x7fcd5c105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 shutdown_connections 2024-05-08T22:10:25.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:25.842+0000 7fcd62841700 1 -- 172.21.15.64:0/2359103060 wait complete. 2024-05-08T22:10:25.844 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-05-08T22:10:25.855 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T22:10:25.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:10:25.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo umount /tmp/cephfs 2024-05-08T22:10:26.194 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:25 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3116475434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:10:26.194 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:25 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/3470276078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:10:26.194 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:25 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2359103060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:10:26.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T22:10:26.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:10:26.196 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-08T22:10:26.301 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:25 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3116475434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:10:26.302 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:25 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/3470276078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:10:26.302 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:25 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2359103060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:10:26.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:10:26.373 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-08T22:10:26.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:25 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3116475434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:10:26.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:25 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/3470276078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:10:26.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:25 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2359103060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:10:26.520 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-08T22:10:26.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:10:26.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-08T22:10:26.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:10:26.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-05-08T22:10:26.973 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:26 smithi064 ceph-mon[42500]: pgmap v734: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-05-08T22:10:27.050 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T22:10:27.060 INFO:tasks.cephadm:Teardown begin 2024-05-08T22:10:27.060 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:10:27.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 -- 172.21.15.64:0/1819096017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef001019a0 msgr2=0x7fef00103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 --2- 172.21.15.64:0/1819096017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef001019a0 0x7fef00103d90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7feefc009990 tx=0x7feefc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 -- 172.21.15.64:0/1819096017 shutdown_connections 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 --2- 172.21.15.64:0/1819096017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef00106c00 0x7fef00108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 --2- 172.21.15.64:0/1819096017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef001019a0 0x7fef00103d90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 --2- 172.21.15.64:0/1819096017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef001042d0 0x7fef001066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 -- 172.21.15.64:0/1819096017 >> 172.21.15.64:0/1819096017 conn(0x7fef000fb2f0 msgr2=0x7fef000fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 -- 172.21.15.64:0/1819096017 shutdown_connections 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 -- 172.21.15.64:0/1819096017 wait complete. 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 Processor -- start 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.220+0000 7fef085c2700 1 -- start start 2024-05-08T22:10:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef085c2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef001019a0 0x7fef00196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef085c2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef001042d0 0x7fef00197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef085c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 0x7fef0019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef085c2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0019e8f0 con 0x7fef00106c00 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef085c2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0019ea30 con 0x7fef001042d0 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef085c2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0019ebc0 con 0x7fef001019a0 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef06b5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 0x7fef0019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef001019a0 0x7fef00196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef06b5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 0x7fef0019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58584/0 (socket says 172.21.15.64:58584) 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef06b5f700 1 -- 172.21.15.64:0/2937686748 learned_addr learned my addr 172.21.15.64:0/2937686748 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef05b5d700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef001042d0 0x7fef00197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 -- 172.21.15.64:0/2937686748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef001042d0 msgr2=0x7fef00197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef001042d0 0x7fef00197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 -- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 msgr2=0x7fef0019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 0x7fef0019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 -- 172.21.15.64:0/2937686748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0095f0 con 0x7fef001019a0 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef06b5f700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 0x7fef0019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.221+0000 7fef0635e700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef001019a0 0x7fef00196dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.222+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc01d070 con 0x7fef001019a0 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.222+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0019ee40 con 0x7fef001019a0 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.222+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feefc021410 con 0x7fef001019a0 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.222+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc021ea0 con 0x7fef001019a0 2024-05-08T22:10:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.222+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0019f200 con 0x7fef001019a0 2024-05-08T22:10:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.223+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef000fce70 con 0x7fef001019a0 2024-05-08T22:10:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.224+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7feefc021580 con 0x7fef001019a0 2024-05-08T22:10:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.224+0000 7feef77fe700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7feeec0a1120 0x7feeec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.225+0000 7fef05b5d700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7feeec0a1120 0x7feeec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.225+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feefc0cd910 con 0x7fef001019a0 2024-05-08T22:10:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.225+0000 7fef05b5d700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7feeec0a1120 0x7feeec0a35e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feef0009c60 tx=0x7feef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:27.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.228+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feefc0970e0 con 0x7fef001019a0 2024-05-08T22:10:27.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:26 smithi160 ceph-mon[46962]: pgmap v734: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-05-08T22:10:27.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:26 smithi186 ceph-mon[46619]: pgmap v734: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-05-08T22:10:27.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.489+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef00067c10 con 0x7fef001019a0 2024-05-08T22:10:27.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.490+0000 7feef77fe700 1 -- 172.21.15.64:0/2937686748 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+10937 (secure 0 0 0) 0x7feefc09a700 con 0x7fef001019a0 2024-05-08T22:10:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:27.490 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T22:10:22.874466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:10:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7feeec0a1120 msgr2=0x7feeec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7feeec0a1120 0x7feeec0a35e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feef0009c60 tx=0x7feef0009260 comp rx=0 tx=0).stop 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef001019a0 msgr2=0x7fef00196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef001019a0 0x7fef00196dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc004730 comp rx=0 tx=0).stop 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 shutdown_connections 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7feeec0a1120 0x7feeec0a35e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef00106c00 0x7fef0019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef001019a0 0x7fef00196dc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 --2- 172.21.15.64:0/2937686748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef001042d0 0x7fef00197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 >> 172.21.15.64:0/2937686748 conn(0x7fef000fb2f0 msgr2=0x7fef00102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 shutdown_connections 2024-05-08T22:10:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.492+0000 7fef085c2700 1 -- 172.21.15.64:0/2937686748 wait complete. 2024-05-08T22:10:27.494 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2024-05-08T22:10:27.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 -- 172.21.15.64:0/861126024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 msgr2=0x7fd6d0073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 --2- 172.21.15.64:0/861126024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0073e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009990 tx=0x7fd6c0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 -- 172.21.15.64:0/861126024 shutdown_connections 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 --2- 172.21.15.64:0/861126024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6d0074f60 0x7fd6d00733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 --2- 172.21.15.64:0/861126024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 0x7fd6d010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 --2- 172.21.15.64:0/861126024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0073e00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 -- 172.21.15.64:0/861126024 >> 172.21.15.64:0/861126024 conn(0x7fd6d00fc400 msgr2=0x7fd6d00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 -- 172.21.15.64:0/861126024 shutdown_connections 2024-05-08T22:10:27.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.632+0000 7fd6d8555700 1 -- 172.21.15.64:0/861126024 wait complete. 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 Processor -- start 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 -- start start 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6d0074f60 0x7fd6d01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 0x7fd6d01a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01a0ac0 con 0x7fd6d0073980 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01a0c00 con 0x7fd6d0074f60 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d8555700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01a0d90 con 0x7fd6d0108d60 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58600/0 (socket says 172.21.15.64:58600) 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 -- 172.21.15.64:0/604336230 learned_addr learned my addr 172.21.15.64:0/604336230 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d5af0700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6d0074f60 0x7fd6d01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d6af2700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 0x7fd6d01a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 -- 172.21.15.64:0/604336230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 msgr2=0x7fd6d01a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 0x7fd6d01a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 -- 172.21.15.64:0/604336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6d0074f60 msgr2=0x7fd6d01994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6d0074f60 0x7fd6d01994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d62f1700 1 -- 172.21.15.64:0/604336230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00095f0 con 0x7fd6d0073980 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.633+0000 7fd6d6af2700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 0x7fd6d01a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.634+0000 7fd6d62f1700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0198f90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00e9a0 tx=0x7fd6cc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.634+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc009790 con 0x7fd6d0073980 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.634+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01a1070 con 0x7fd6d0073980 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.634+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6cc00cbd0 con 0x7fd6d0073980 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.634+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc018680 con 0x7fd6d0073980 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.634+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01a15d0 con 0x7fd6d0073980 2024-05-08T22:10:27.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.635+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d0107b70 con 0x7fd6d0073980 2024-05-08T22:10:27.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.636+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd6cc00cd40 con 0x7fd6d0073980 2024-05-08T22:10:27.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.637+0000 7fd6c77fe700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6bc0a1120 0x7fd6bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:27.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.637+0000 7fd6d5af0700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6bc0a1120 0x7fd6bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:27.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.637+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd6cc014070 con 0x7fd6d0073980 2024-05-08T22:10:27.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.637+0000 7fd6d5af0700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6bc0a1120 0x7fd6bc0a35e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b2e0 tx=0x7fd6c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:27.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.640+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd6cc096860 con 0x7fd6d0073980 2024-05-08T22:10:27.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.899+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6d010e860 con 0x7fd6d0073980 2024-05-08T22:10:27.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.899+0000 7fd6c77fe700 1 -- 172.21.15.64:0/604336230 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+10937 (secure 0 0 0) 0x7fd6cc099e80 con 0x7fd6d0073980 2024-05-08T22:10:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:27.900 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1013310414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1013310414},{"type":"v1","addr":"172.21.15.64:6835","nonce":1013310414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2682658639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2682658639},{"type":"v1","addr":"172.21.15.64:6837","nonce":2682658639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:53:23.555551+0000","modified":"2024-05-08T22:10:22.874466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24484,"mds_2":24476,"mds_3":24427,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/3760541428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3760541428},{"type":"v1","addr":"172.21.15.64:6839","nonce":3760541428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1669365277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1669365277},{"type":"v1","addr":"172.21.15.64:6841","nonce":1669365277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.160:6833/3542328594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3542328594},{"type":"v1","addr":"172.21.15.160:6833","nonce":3542328594}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24427":{"gid":24427,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6833/1716244539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1716244539},{"type":"v1","addr":"172.21.15.186:6833","nonce":1716244539}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/770617410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":770617410},{"type":"v1","addr":"172.21.15.160:6835","nonce":770617410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/4069999836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4069999836},{"type":"v1","addr":"172.21.15.186:6835","nonce":4069999836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/3888486173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3888486173},{"type":"v1","addr":"172.21.15.186:6837","nonce":3888486173}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/4233185014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":4233185014},{"type":"v1","addr":"172.21.15.160:6837","nonce":4233185014}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/539285459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":539285459},{"type":"v1","addr":"172.21.15.186:6839","nonce":539285459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2466892222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2466892222},{"type":"v1","addr":"172.21.15.160:6839","nonce":2466892222}]},"join_fscid":-1,"export_targets":[2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6bc0a1120 msgr2=0x7fd6bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6bc0a1120 0x7fd6bc0a35e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b2e0 tx=0x7fd6c0005fb0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 msgr2=0x7fd6d0198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0198f90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00e9a0 tx=0x7fd6cc00ed60 comp rx=0 tx=0).stop 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 shutdown_connections 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fd6bc0a1120 0x7fd6bc0a35e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6d0108d60 0x7fd6d01a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6d0073980 0x7fd6d0198f90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 --2- 172.21.15.64:0/604336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6d0074f60 0x7fd6d01994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 >> 172.21.15.64:0/604336230 conn(0x7fd6d00fc400 msgr2=0x7fd6d0104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 shutdown_connections 2024-05-08T22:10:27.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:27.902+0000 7fd6d8555700 1 -- 172.21.15.64:0/604336230 wait complete. 2024-05-08T22:10:27.904 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2024-05-08T22:10:27.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 -- 172.21.15.64:0/1567449158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504104240 msgr2=0x7f3504106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 --2- 172.21.15.64:0/1567449158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504104240 0x7f3504106630 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3500009990 tx=0x7f3500009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 -- 172.21.15.64:0/1567449158 shutdown_connections 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 --2- 172.21.15.64:0/1567449158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3504106b70 0x7f3504108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 --2- 172.21.15.64:0/1567449158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504101910 0x7f3504103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 --2- 172.21.15.64:0/1567449158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504104240 0x7f3504106630 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.044+0000 7f350c64c700 1 -- 172.21.15.64:0/1567449158 >> 172.21.15.64:0/1567449158 conn(0x7f35040fb300 msgr2=0x7f35040fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 -- 172.21.15.64:0/1567449158 shutdown_connections 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 -- 172.21.15.64:0/1567449158 wait complete. 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 Processor -- start 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 -- start start 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3504101910 0x7f3504194bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504104240 0x7f3504195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 0x7f350419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350419c6f0 con 0x7f3504106b70 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350419c830 con 0x7f3504101910 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350c64c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350419c9c0 con 0x7f3504104240 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350abe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 0x7f350419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350abe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 0x7f350419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58620/0 (socket says 172.21.15.64:58620) 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.045+0000 7f350abe9700 1 -- 172.21.15.64:0/1644298797 learned_addr learned my addr 172.21.15.64:0/1644298797 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350a3e8700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3504101910 0x7f3504194bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f3509be7700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504104240 0x7f3504195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350abe9700 1 -- 172.21.15.64:0/1644298797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504104240 msgr2=0x7f3504195100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350abe9700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504104240 0x7f3504195100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350abe9700 1 -- 172.21.15.64:0/1644298797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3504101910 msgr2=0x7f3504194bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350abe9700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3504101910 0x7f3504194bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350abe9700 1 -- 172.21.15.64:0/1644298797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000095f0 con 0x7f3504106b70 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f3509be7700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504104240 0x7f3504195100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350abe9700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 0x7f350419c120 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00ef30 tx=0x7f34fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc018070 con 0x7f3504106b70 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350419cc40 con 0x7f3504106b70 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc00f040 con 0x7f3504106b70 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00cdb0 con 0x7f3504106b70 2024-05-08T22:10:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.046+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350419d150 con 0x7f3504106b70 2024-05-08T22:10:28.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.047+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35040fce80 con 0x7f3504106b70 2024-05-08T22:10:28.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.048+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34fc003a00 con 0x7f3504106b70 2024-05-08T22:10:28.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.049+0000 7f34fb7fe700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34f00a1120 0x7f34f00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.049+0000 7f350a3e8700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34f00a1120 0x7f34f00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.049+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34fc0cce60 con 0x7f3504106b70 2024-05-08T22:10:28.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.050+0000 7f350a3e8700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34f00a1120 0x7f34f00a35e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005950 tx=0x7f34f400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:28.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.053+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f34fc096690 con 0x7f3504106b70 2024-05-08T22:10:28.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2937686748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:27 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/604336230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.264+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3504067380 con 0x7f3504106b70 2024-05-08T22:10:28.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.266+0000 7f34fb7fe700 1 -- 172.21.15.64:0/1644298797 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22495 (secure 0 0 0) 0x7f34fc099cb0 con 0x7f3504106b70 2024-05-08T22:10:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-08T22:10:28.267 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":81,"fsid":"71e641de-0d84-11ef-bc97-c7b262605968","created":"2024-05-08T21:47:28.948985+0000","modified":"2024-05-08T21:54:21.242448+0000","last_up_change":"2024-05-08T21:52:48.528814+0000","last_in_change":"2024-05-08T21:52:30.975355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:50:08.557347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:53:21.578075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:53:22.933075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1bb0a350-33bc-4f0d-a340-96bb54726c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6803","nonce":4148876663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6805","nonce":4148876663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6809","nonce":4148876663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4148876663},{"type":"v1","addr":"172.21.15.64:6807","nonce":4148876663}]},"public_addr":"172.21.15.64:6803/4148876663","cluster_addr":"172.21.15.64:6805/4148876663","heartbeat_back_addr":"172.21.15.64:6809/4148876663","heartbeat_front_addr":"172.21.15.64:6807/4148876663","state":["exists","up"]},{"osd":1,"uuid":"0fb69147-6892-4a82-8c67-94919896a9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6801","nonce":255573716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6803","nonce":255573716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6807","nonce":255573716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":255573716},{"type":"v1","addr":"172.21.15.160:6805","nonce":255573716}]},"public_addr":"172.21.15.160:6801/255573716","cluster_addr":"172.21.15.160:6803/255573716","heartbeat_back_addr":"172.21.15.160:6807/255573716","heartbeat_front_addr":"172.21.15.160:6805/255573716","state":["exists","up"]},{"osd":2,"uuid":"65e106b3-aa74-4293-98f6-09717bbfca26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6801","nonce":348681043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6803","nonce":348681043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6807","nonce":348681043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":348681043},{"type":"v1","addr":"172.21.15.186:6805","nonce":348681043}]},"public_addr":"172.21.15.186:6801/348681043","cluster_addr":"172.21.15.186:6803/348681043","heartbeat_back_addr":"172.21.15.186:6807/348681043","heartbeat_front_addr":"172.21.15.186:6805/348681043","state":["exists","up"]},{"osd":3,"uuid":"6a840d5a-a687-4cf2-a946-7a2f46bdfd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6811","nonce":3390967060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6813","nonce":3390967060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6817","nonce":3390967060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3390967060},{"type":"v1","addr":"172.21.15.64:6815","nonce":3390967060}]},"public_addr":"172.21.15.64:6811/3390967060","cluster_addr":"172.21.15.64:6813/3390967060","heartbeat_back_addr":"172.21.15.64:6817/3390967060","heartbeat_front_addr":"172.21.15.64:6815/3390967060","state":["exists","up"]},{"osd":4,"uuid":"e00f8790-f664-4e52-b320-ab5e454f0235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6809","nonce":1821714303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6811","nonce":1821714303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6815","nonce":1821714303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1821714303},{"type":"v1","addr":"172.21.15.160:6813","nonce":1821714303}]},"public_addr":"172.21.15.160:6809/1821714303","cluster_addr":"172.21.15.160:6811/1821714303","heartbeat_back_addr":"172.21.15.160:6815/1821714303","heartbeat_front_addr":"172.21.15.160:6813/1821714303","state":["exists","up"]},{"osd":5,"uuid":"7e44e11d-06e2-40ef-9bf4-43bfd19c99da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6809","nonce":1348911288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6811","nonce":1348911288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6815","nonce":1348911288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1348911288},{"type":"v1","addr":"172.21.15.186:6813","nonce":1348911288}]},"public_addr":"172.21.15.186:6809/1348911288","cluster_addr":"172.21.15.186:6811/1348911288","heartbeat_back_addr":"172.21.15.186:6815/1348911288","heartbeat_front_addr":"172.21.15.186:6813/1348911288","state":["exists","up"]},{"osd":6,"uuid":"e46cfa92-0895-463b-92e8-a8ab73a0d1e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6819","nonce":3400846142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6821","nonce":3400846142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6825","nonce":3400846142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3400846142},{"type":"v1","addr":"172.21.15.64:6823","nonce":3400846142}]},"public_addr":"172.21.15.64:6819/3400846142","cluster_addr":"172.21.15.64:6821/3400846142","heartbeat_back_addr":"172.21.15.64:6825/3400846142","heartbeat_front_addr":"172.21.15.64:6823/3400846142","state":["exists","up"]},{"osd":7,"uuid":"9dc1fffb-253b-47fb-99de-85b4c9cfa6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6817","nonce":3316502570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6819","nonce":3316502570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6823","nonce":3316502570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3316502570},{"type":"v1","addr":"172.21.15.160:6821","nonce":3316502570}]},"public_addr":"172.21.15.160:6817/3316502570","cluster_addr":"172.21.15.160:6819/3316502570","heartbeat_back_addr":"172.21.15.160:6823/3316502570","heartbeat_front_addr":"172.21.15.160:6821/3316502570","state":["exists","up"]},{"osd":8,"uuid":"316f22c0-33f7-44c2-ae4f-0ed122d52d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6817","nonce":4072780638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6819","nonce":4072780638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6823","nonce":4072780638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4072780638},{"type":"v1","addr":"172.21.15.186:6821","nonce":4072780638}]},"public_addr":"172.21.15.186:6817/4072780638","cluster_addr":"172.21.15.186:6819/4072780638","heartbeat_back_addr":"172.21.15.186:6823/4072780638","heartbeat_front_addr":"172.21.15.186:6821/4072780638","state":["exists","up"]},{"osd":9,"uuid":"0bbe8d1f-ece5-435b-89db-1abeefdb4320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6827","nonce":200787259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6829","nonce":200787259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6833","nonce":200787259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":200787259},{"type":"v1","addr":"172.21.15.64:6831","nonce":200787259}]},"public_addr":"172.21.15.64:6827/200787259","cluster_addr":"172.21.15.64:6829/200787259","heartbeat_back_addr":"172.21.15.64:6833/200787259","heartbeat_front_addr":"172.21.15.64:6831/200787259","state":["exists","up"]},{"osd":10,"uuid":"63c6380e-f30e-4139-a9a3-6250bb4c4358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6825","nonce":1117750760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6827","nonce":1117750760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6831","nonce":1117750760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1117750760},{"type":"v1","addr":"172.21.15.160:6829","nonce":1117750760}]},"public_addr":"172.21.15.160:6825/1117750760","cluster_addr":"172.21.15.160:6827/1117750760","heartbeat_back_addr":"172.21.15.160:6831/1117750760","heartbeat_front_addr":"172.21.15.160:6829/1117750760","state":["exists","up"]},{"osd":11,"uuid":"a05f3067-983c-4656-9914-05ab90f5bd9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6825","nonce":2726552436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6827","nonce":2726552436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6831","nonce":2726552436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2726552436},{"type":"v1","addr":"172.21.15.186:6829","nonce":2726552436}]},"public_addr":"172.21.15.186:6825/2726552436","cluster_addr":"172.21.15.186:6827/2726552436","heartbeat_back_addr":"172.21.15.186:6831/2726552436","heartbeat_front_addr":"172.21.15.186:6829/2726552436","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:31.359152+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:49:48.939330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:06.218065+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:23.577486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:41.232131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:50:58.555959+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:16.574063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:34.506730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:51:51.717716+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:09.526571+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:27.694566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:52:45.815491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1096250282":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6801/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:6800/865555054":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/2750019664":"2024-05-09T21:48:20.034470+0000","172.21.15.64:0/3310656394":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6800/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/1642224217":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3843761611":"2024-05-09T21:47:59.744672+0000","172.21.15.64:6801/1406677494":"2024-05-09T21:47:59.744672+0000","172.21.15.64:0/3369547289":"2024-05-09T21:48:20.034470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34f00a1120 msgr2=0x7f34f00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34f00a1120 0x7f34f00a35e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005950 tx=0x7f34f400b700 comp rx=0 tx=0).stop 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 msgr2=0x7f350419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 0x7f350419c120 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00ef30 tx=0x7f34fc00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 shutdown_connections 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f34f00a1120 0x7f34f00a35e0 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005950 tx=0x7f34f400b700 comp rx=0 tx=0).stop 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3504101910 0x7f3504194bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504106b70 0x7f350419c120 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 --2- 172.21.15.64:0/1644298797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504104240 0x7f3504195100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 >> 172.21.15.64:0/1644298797 conn(0x7f35040fb300 msgr2=0x7f3504103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:28.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 shutdown_connections 2024-05-08T22:10:28.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.268+0000 7f350c64c700 1 -- 172.21.15.64:0/1644298797 wait complete. 2024-05-08T22:10:28.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:10:28.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2937686748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:27 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/604336230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2937686748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:27 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/604336230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 -- 172.21.15.64:0/3204325778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e40ffe80 msgr2=0x7f23e41002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/3204325778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e40ffe80 0x7f23e41002a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009990 tx=0x7f23e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 -- 172.21.15.64:0/3204325778 shutdown_connections 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/3204325778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e4101990 0x7f23e41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/3204325778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e40ffe80 0x7f23e41002a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/3204325778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e4100fd0 0x7f23e4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 -- 172.21.15.64:0/3204325778 >> 172.21.15.64:0/3204325778 conn(0x7f23e40fb3f0 msgr2=0x7f23e40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 -- 172.21.15.64:0/3204325778 shutdown_connections 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.409+0000 7f23eb3cd700 1 -- 172.21.15.64:0/3204325778 wait complete. 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 Processor -- start 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 -- start start 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e40ffe80 0x7f23e4196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e4100fd0 0x7f23e4197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 0x7f23e419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e419eb10 con 0x7f23e4101990 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e419ec50 con 0x7f23e40ffe80 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23eb3cd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e419ede0 con 0x7f23e4100fd0 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23e996a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 0x7f23e419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23e996a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 0x7f23e419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58642/0 (socket says 172.21.15.64:58642) 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.410+0000 7f23e996a700 1 -- 172.21.15.64:0/2124686066 learned_addr learned my addr 172.21.15.64:0/2124686066 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e9169700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e40ffe80 0x7f23e4196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e8968700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e4100fd0 0x7f23e4197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e996a700 1 -- 172.21.15.64:0/2124686066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e4100fd0 msgr2=0x7f23e4197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e996a700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e4100fd0 0x7f23e4197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e996a700 1 -- 172.21.15.64:0/2124686066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e40ffe80 msgr2=0x7f23e4196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e996a700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e40ffe80 0x7f23e4196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e996a700 1 -- 172.21.15.64:0/2124686066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00095f0 con 0x7f23e4101990 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23e996a700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 0x7f23e419e540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00ada0 tx=0x7f23dc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00f070 con 0x7f23e4101990 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23dc007b30 con 0x7f23e4101990 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e419f0c0 con 0x7f23e4101990 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0053e0 con 0x7f23e4101990 2024-05-08T22:10:28.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.411+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e419f5d0 con 0x7f23e4101990 2024-05-08T22:10:28.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.412+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e4067460 con 0x7f23e4101990 2024-05-08T22:10:28.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.414+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f23dc00c980 con 0x7f23e4101990 2024-05-08T22:10:28.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.415+0000 7f23da7fc700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23d00a11a0 0x7f23d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.415+0000 7f23e9169700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23d00a11a0 0x7f23d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.415+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23dc0cc4a0 con 0x7f23e4101990 2024-05-08T22:10:28.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.415+0000 7f23e9169700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23d00a11a0 0x7f23d00a3660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23e000b5c0 tx=0x7f23e0003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:28.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.418+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f23dc095c70 con 0x7f23e4101990 2024-05-08T22:10:28.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.673+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f23e410b950 con 0x7f23e4101990 2024-05-08T22:10:28.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.687+0000 7f23da7fc700 1 -- 172.21.15.64:0/2124686066 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f23dc099290 con 0x7f23e4101990 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23d00a11a0 msgr2=0x7f23d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23d00a11a0 0x7f23d00a3660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23e000b5c0 tx=0x7f23e0003790 comp rx=0 tx=0).stop 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 msgr2=0x7f23e419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 0x7f23e419e540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00ada0 tx=0x7f23dc00c330 comp rx=0 tx=0).stop 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 shutdown_connections 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f23d00a11a0 0x7f23d00a3660 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e40ffe80 0x7f23e4196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e4101990 0x7f23e419e540 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 --2- 172.21.15.64:0/2124686066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e4100fd0 0x7f23e4197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 >> 172.21.15.64:0/2124686066 conn(0x7f23e40fb3f0 msgr2=0x7f23e40692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 shutdown_connections 2024-05-08T22:10:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.690+0000 7f23eb3cd700 1 -- 172.21.15.64:0/2124686066 wait complete. 2024-05-08T22:10:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:10:28.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:10:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 -- 172.21.15.64:0/2626347271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c0ff220 msgr2=0x7f813c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 --2- 172.21.15.64:0/2626347271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c0ff220 0x7f813c0ff640 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f812c009990 tx=0x7f812c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 -- 172.21.15.64:0/2626347271 shutdown_connections 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 --2- 172.21.15.64:0/2626347271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f813c102b90 0x7f813c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 --2- 172.21.15.64:0/2626347271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c0ff220 0x7f813c0ff640 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 --2- 172.21.15.64:0/2626347271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c1021f0 0x7f813c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 -- 172.21.15.64:0/2626347271 >> 172.21.15.64:0/2626347271 conn(0x7f813c0fab50 msgr2=0x7f813c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 -- 172.21.15.64:0/2626347271 shutdown_connections 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.839+0000 7f81424ac700 1 -- 172.21.15.64:0/2626347271 wait complete. 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 Processor -- start 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 -- start start 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c0ff220 0x7f813c101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f813c1021f0 0x7f813c1001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 0x7f813c1006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c100e80 con 0x7f813c102b90 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c100ff0 con 0x7f813c1021f0 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f81424ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c1a0f40 con 0x7f813c0ff220 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f8140a49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 0x7f813c1006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f8140a49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 0x7f813c1006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58714/0 (socket says 172.21.15.64:58714) 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.840+0000 7f8140a49700 1 -- 172.21.15.64:0/786772115 learned_addr learned my addr 172.21.15.64:0/786772115 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f813bfff700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c0ff220 0x7f813c101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f813b7fe700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f813c1021f0 0x7f813c1001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f8140a49700 1 -- 172.21.15.64:0/786772115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c0ff220 msgr2=0x7f813c101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f8140a49700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c0ff220 0x7f813c101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f8140a49700 1 -- 172.21.15.64:0/786772115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f813c1021f0 msgr2=0x7f813c1001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f8140a49700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f813c1021f0 0x7f813c1001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f8140a49700 1 -- 172.21.15.64:0/786772115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134009c40 con 0x7f813c102b90 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f813bfff700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c0ff220 0x7f813c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f8140a49700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 0x7f813c1006e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f813400c370 tx=0x7f813400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134017070 con 0x7f813c102b90 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0095f0 con 0x7f813c102b90 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f813400f040 con 0x7f813c102b90 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400e050 con 0x7f813c102b90 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.841+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c1a14f0 con 0x7f813c102b90 2024-05-08T22:10:28.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.842+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c1084d0 con 0x7f813c102b90 2024-05-08T22:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.843+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f813400cb00 con 0x7f813c102b90 2024-05-08T22:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.844+0000 7f81397fa700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f81240a1120 0x7f81240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.844+0000 7f813bfff700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f81240a1120 0x7f81240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.844+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8641+0+0 (secure 0 0 0) 0x7f81340cc430 con 0x7f813c102b90 2024-05-08T22:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.844+0000 7f813bfff700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f81240a1120 0x7f81240a35e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f812c00b680 tx=0x7f812c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:28.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:28.848+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f81340959c0 con 0x7f813c102b90 2024-05-08T22:10:28.963 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a[42476]: 2024-05-08T22:10:28.674+0000 7f90e5102700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:10:28.963 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a[94879]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:28.963 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a[94879]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:28.963 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mds[94902]: -- 172.21.15.64:0/990004840 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f84215e1a0 con 0x55f842134800 2024-05-08T22:10:28.963 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a[94879]: starting mds.a at 2024-05-08T22:10:28.964 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g[94493]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:28.964 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g[94493]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:28.964 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mds[94516]: -- 172.21.15.64:0/3135307555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625feb6a1a0 con 0x5625feb41000 2024-05-08T22:10:28.964 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:10:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g[94493]: starting mds.g at 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/1644298797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2124686066' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/2124686066' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T22:10:29.045 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mon[46962]: pgmap v736: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: starting mds.b at 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e[82283]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e[82283]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mds[82306]: -- 172.21.15.160:0/1832959124 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aa6feaa1a0 con 0x55aa6fe80800 2024-05-08T22:10:29.046 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e[82283]: starting mds.e at 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h[82623]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h[82623]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h[82623]: starting mds.h at 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k[81932]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k[81932]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:10:28 smithi160 ceph-mds[81955]: -- 172.21.15.160:0/1061063283 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f7d4f6e1a0 con 0x55f7d4f44c00 2024-05-08T22:10:29.047 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:10:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k[81932]: starting mds.k at 2024-05-08T22:10:29.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.109+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f813c107eb0 con 0x7f813c102b90 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l[82639]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l[82639]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l[82639]: starting mds.l at 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c[83335]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c[83335]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mds[83358]: -- 172.21.15.186:0/257685404 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560336f681a0 con 0x560336f3f000 2024-05-08T22:10:29.142 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c[83335]: starting mds.c at 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f[82984]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f[82984]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mds[83007]: -- 172.21.15.186:0/3763533346 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56543fa661a0 con 0x56543fa3c800 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f[82984]: starting mds.f at 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i[83678]: ignoring --setuser ceph since I am not root 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i[83678]: ignoring --setgroup ceph since I am not root 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mds[83701]: -- 172.21.15.186:0/1807963473 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557364b4c1a0 con 0x557364b22c00 2024-05-08T22:10:29.143 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:10:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i[83678]: starting mds.i at 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/1644298797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2124686066' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/2124686066' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T22:10:29.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:28 smithi186 ceph-mon[46619]: pgmap v736: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/1644298797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2124686066' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/2124686066' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T22:10:29.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:28 smithi064 ceph-mon[42500]: pgmap v736: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-08T22:10:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.692+0000 7f81397fa700 1 -- 172.21.15.64:0/786772115 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8134098fe0 con 0x7f813c102b90 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.696+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f81240a1120 msgr2=0x7f81240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.696+0000 7f81424ac700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f81240a1120 0x7f81240a35e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f812c00b680 tx=0x7f812c005fd0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.696+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 msgr2=0x7f813c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.696+0000 7f81424ac700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 0x7f813c1006e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f813400c370 tx=0x7f813400c680 comp rx=0 tx=0).stop 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 shutdown_connections 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f81240a1120 0x7f81240a35e0 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f812c00b680 tx=0x7f812c005fd0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f813c102b90 0x7f813c1006e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c0ff220 0x7f813c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 --2- 172.21.15.64:0/786772115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f813c1021f0 0x7f813c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 >> 172.21.15.64:0/786772115 conn(0x7f813c0fab50 msgr2=0x7f813c105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 shutdown_connections 2024-05-08T22:10:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.697+0000 7f81424ac700 1 -- 172.21.15.64:0/786772115 wait complete. 2024-05-08T22:10:29.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:10:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 -- 172.21.15.64:0/506467788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980ffe60 msgr2=0x7fce98100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/506467788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980ffe60 0x7fce98100280 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fce88009990 tx=0x7fce88009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 -- 172.21.15.64:0/506467788 shutdown_connections 2024-05-08T22:10:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/506467788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce98101970 0x7fce98108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/506467788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980ffe60 0x7fce98100280 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/506467788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce98100fb0 0x7fce98101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 -- 172.21.15.64:0/506467788 >> 172.21.15.64:0/506467788 conn(0x7fce980fb3f0 msgr2=0x7fce980fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 -- 172.21.15.64:0/506467788 shutdown_connections 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 -- 172.21.15.64:0/506467788 wait complete. 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 Processor -- start 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.851+0000 7fce9d7b6700 1 -- start start 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce9d7b6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce980ffe60 0x7fce98196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce9d7b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 0x7fce981973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce9d7b6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce98101970 0x7fce9819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce9d7b6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9819e9e0 con 0x7fce98100fb0 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce9d7b6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9819eb20 con 0x7fce98101970 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce9d7b6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9819ecb0 con 0x7fce980ffe60 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 0x7fce981973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:29.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 0x7fce981973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58730/0 (socket says 172.21.15.64:58730) 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 -- 172.21.15.64:0/810761976 learned_addr learned my addr 172.21.15.64:0/810761976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce96ffd700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce980ffe60 0x7fce98196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce977fe700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce98101970 0x7fce9819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 -- 172.21.15.64:0/810761976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce980ffe60 msgr2=0x7fce98196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce980ffe60 0x7fce98196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 -- 172.21.15.64:0/810761976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce98101970 msgr2=0x7fce9819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce98101970 0x7fce9819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 -- 172.21.15.64:0/810761976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880095f0 con 0x7fce98100fb0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce967fc700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 0x7fce981973f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d950 tx=0x7fce8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00f890 con 0x7fce98100fb0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.852+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce8c010460 con 0x7fce98100fb0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.853+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9819ef90 con 0x7fce98100fb0 2024-05-08T22:10:29.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.853+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00e610 con 0x7fce98100fb0 2024-05-08T22:10:29.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.853+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9819f410 con 0x7fce98100fb0 2024-05-08T22:10:29.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.854+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce98068140 con 0x7fce98100fb0 2024-05-08T22:10:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.855+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fce8c0108d0 con 0x7fce98100fb0 2024-05-08T22:10:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.855+0000 7fce7ffff700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fce800a1120 0x7fce800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.856+0000 7fce96ffd700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fce800a1120 0x7fce800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.856+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8641+0+0 (secure 0 0 0) 0x7fce8c0cc810 con 0x7fce98100fb0 2024-05-08T22:10:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.856+0000 7fce96ffd700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fce800a1120 0x7fce800a35e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b2e0 tx=0x7fce8800b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:29.860+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fce8c095e00 con 0x7fce98100fb0 2024-05-08T22:10:30.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.071+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce9819f750 con 0x7fce98100fb0 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/786772115' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: Cluster is now healthy 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6833/1914499109,v1:172.21.15.186:6835/1914499109] up:boot 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.64:6838/4219028669,v1:172.21.15.64:6840/4219028669] up:boot 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.64:6839/1937257767,v1:172.21.15.64:6841/1937257767] up:boot 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6836/444194566,v1:172.21.15.186:6837/444194566] up:boot 2024-05-08T22:10:30.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6832/4293645143,v1:172.21.15.186:6834/4293645143] up:boot 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.186:6838/3769625944,v1:172.21.15.186:6839/3769625944] up:boot 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6832/1980989486,v1:172.21.15.160:6833/1980989486] up:boot 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6838/567969711,v1:172.21.15.160:6839/567969711] up:boot 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6834/352565225,v1:172.21.15.160:6835/352565225] up:boot 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: mds.? [v2:172.21.15.160:6836/1721278074,v1:172.21.15.160:6837/1721278074] up:boot 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/786772115' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: fsmap 12 up:standby 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:10:30.296 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:10:30.297 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:10:30.297 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:10:30.297 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:29 smithi160 ceph-mon[46962]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/786772115' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: Cluster is now healthy 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6833/1914499109,v1:172.21.15.186:6835/1914499109] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.64:6838/4219028669,v1:172.21.15.64:6840/4219028669] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.64:6839/1937257767,v1:172.21.15.64:6841/1937257767] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6836/444194566,v1:172.21.15.186:6837/444194566] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6832/4293645143,v1:172.21.15.186:6834/4293645143] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.186:6838/3769625944,v1:172.21.15.186:6839/3769625944] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6832/1980989486,v1:172.21.15.160:6833/1980989486] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6838/567969711,v1:172.21.15.160:6839/567969711] up:boot 2024-05-08T22:10:30.392 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6834/352565225,v1:172.21.15.160:6835/352565225] up:boot 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: mds.? [v2:172.21.15.160:6836/1721278074,v1:172.21.15.160:6837/1721278074] up:boot 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/786772115' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: fsmap 12 up:standby 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:10:30.393 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:29 smithi186 ceph-mon[46619]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/786772115' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: Cluster is now healthy 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6833/1914499109,v1:172.21.15.186:6835/1914499109] up:boot 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.64:6838/4219028669,v1:172.21.15.64:6840/4219028669] up:boot 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.64:6839/1937257767,v1:172.21.15.64:6841/1937257767] up:boot 2024-05-08T22:10:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6836/444194566,v1:172.21.15.186:6837/444194566] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6832/4293645143,v1:172.21.15.186:6834/4293645143] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.186:6838/3769625944,v1:172.21.15.186:6839/3769625944] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6832/1980989486,v1:172.21.15.160:6833/1980989486] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6838/567969711,v1:172.21.15.160:6839/567969711] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6834/352565225,v1:172.21.15.160:6835/352565225] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: mds.? [v2:172.21.15.160:6836/1721278074,v1:172.21.15.160:6837/1721278074] up:boot 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/786772115' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: fsmap 12 up:standby 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:10:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:10:30.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:10:30.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:10:30.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:29 smithi064 ceph-mon[42500]: from='mgr.14152 172.21.15.64:0/3119531605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:10:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.983+0000 7fce7ffff700 1 -- 172.21.15.64:0/810761976 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce8c0160d0 con 0x7fce98100fb0 2024-05-08T22:10:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fce800a1120 msgr2=0x7fce800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fce800a1120 0x7fce800a35e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b2e0 tx=0x7fce8800b6f0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 msgr2=0x7fce981973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 0x7fce981973f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d950 tx=0x7fce8c00dc60 comp rx=0 tx=0).stop 2024-05-08T22:10:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 shutdown_connections 2024-05-08T22:10:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7fce800a1120 0x7fce800a35e0 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b2e0 tx=0x7fce8800b6f0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce98101970 0x7fce9819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce980ffe60 0x7fce98196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 --2- 172.21.15.64:0/810761976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce98100fb0 0x7fce981973f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 >> 172.21.15.64:0/810761976 conn(0x7fce980fb3f0 msgr2=0x7fce98069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.988+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 shutdown_connections 2024-05-08T22:10:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:30.989+0000 7fce9d7b6700 1 -- 172.21.15.64:0/810761976 wait complete. 2024-05-08T22:10:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:10:31.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:10:31.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 -- 172.21.15.64:0/404491355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102b10 msgr2=0x7f7cf81093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/404491355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102b10 0x7f7cf81093c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000b0d0 tx=0x7f7cf000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 -- 172.21.15.64:0/404491355 shutdown_connections 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/404491355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf8100f80 0x7f7cf81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/404491355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102b10 0x7f7cf81093c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/404491355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cf8102170 0x7f7cf81025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 -- 172.21.15.64:0/404491355 >> 172.21.15.64:0/404491355 conn(0x7f7cf80fc4f0 msgr2=0x7f7cf80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 -- 172.21.15.64:0/404491355 shutdown_connections 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 -- 172.21.15.64:0/404491355 wait complete. 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 Processor -- start 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 -- start start 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf8100f80 0x7f7cf8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.143+0000 7f7cffaf8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 0x7f7cf81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cffaf8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cf8102b10 0x7f7cf81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cffaf8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81a0c90 con 0x7f7cf8102170 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cffaf8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81a0dd0 con 0x7f7cf8100f80 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cffaf8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81a0f60 con 0x7f7cf8102b10 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 0x7f7cf81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 0x7f7cf81996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58754/0 (socket says 172.21.15.64:58754) 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 -- 172.21.15.64:0/233366887 learned_addr learned my addr 172.21.15.64:0/233366887 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd894700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf8100f80 0x7f7cf8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfe095700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cf8102b10 0x7f7cf81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 -- 172.21.15.64:0/233366887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cf8102b10 msgr2=0x7f7cf81a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cf8102b10 0x7f7cf81a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 -- 172.21.15.64:0/233366887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf8100f80 msgr2=0x7f7cf8199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf8100f80 0x7f7cf8199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 -- 172.21.15.64:0/233366887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf0009d00 con 0x7f7cf8102170 2024-05-08T22:10:31.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cfd093700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 0x7f7cf81996a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e9a0 tx=0x7f7ce800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:31.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8009790 con 0x7f7cf8102170 2024-05-08T22:10:31.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ce8010430 con 0x7f7cf8102170 2024-05-08T22:10:31.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf81a1240 con 0x7f7cf8102170 2024-05-08T22:10:31.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8018680 con 0x7f7cf8102170 2024-05-08T22:10:31.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.144+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf81a1750 con 0x7f7cf8102170 2024-05-08T22:10:31.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.145+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf8067cf0 con 0x7f7cf8102170 2024-05-08T22:10:31.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.146+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7ce80105a0 con 0x7f7cf8102170 2024-05-08T22:10:31.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.147+0000 7f7ceeffd700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7ce40a1120 0x7f7ce40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:31.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.147+0000 7f7cfd894700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7ce40a1120 0x7f7ce40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:31.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.148+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8116+0+0 (secure 0 0 0) 0x7f7ce8014070 con 0x7f7cf8102170 2024-05-08T22:10:31.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.148+0000 7f7cfd894700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7ce40a1120 0x7f7ce40a35e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8101fd0 tx=0x7f7cf400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:31.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.151+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ce8096450 con 0x7f7cf8102170 2024-05-08T22:10:31.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:30 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/810761976' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:10:31.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:30 smithi160 ceph-mon[46962]: pgmap v737: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-05-08T22:10:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.363+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7cf810c7a0 con 0x7f7cf8102170 2024-05-08T22:10:31.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:30 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/810761976' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:10:31.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:30 smithi186 ceph-mon[46619]: pgmap v737: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-05-08T22:10:31.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:30 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/810761976' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:10:31.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:30 smithi064 ceph-mon[42500]: pgmap v737: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-05-08T22:10:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.984+0000 7f7ceeffd700 1 -- 172.21.15.64:0/233366887 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7ce8026020 con 0x7f7cf8102170 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7ce40a1120 msgr2=0x7f7ce40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7ce40a1120 0x7f7ce40a35e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8101fd0 tx=0x7f7cf400b410 comp rx=0 tx=0).stop 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 msgr2=0x7f7cf81996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 0x7f7cf81996a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e9a0 tx=0x7f7ce800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 shutdown_connections 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:6800/3946872331,v1:172.21.15.64:6801/3946872331] conn(0x7f7ce40a1120 0x7f7ce40a35e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf8100f80 0x7f7cf8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cf8102b10 0x7f7cf81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 --2- 172.21.15.64:0/233366887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cf8102170 0x7f7cf81996a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 >> 172.21.15.64:0/233366887 conn(0x7f7cf80fc4f0 msgr2=0x7f7cf80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 shutdown_connections 2024-05-08T22:10:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-08T22:10:31.987+0000 7f7cffaf8700 1 -- 172.21.15.64:0/233366887 wait complete. 2024-05-08T22:10:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-05-08T22:10:32.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:10:32.035 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:10:32.111 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:10:32.156 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T22:10:32.157 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:10:32.176 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:10:32.193 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:10:32.209 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T22:10:32.209 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T22:10:32.210 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a 2024-05-08T22:10:32.252 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:31 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/810761976' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:10:32.252 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:31 smithi064 ceph-mon[42500]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:10:32.252 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:31 smithi064 ceph-mon[42500]: from='client.? 172.21.15.64:0/233366887' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:10:32.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:31 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/810761976' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:10:32.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:31 smithi160 ceph-mon[46962]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:10:32.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:31 smithi160 ceph-mon[46962]: from='client.? 172.21.15.64:0/233366887' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:10:32.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:31 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/810761976' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:10:32.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:31 smithi186 ceph-mon[46619]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:10:32.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:31 smithi186 ceph-mon[46619]: from='client.? 172.21.15.64:0/233366887' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:10:32.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:32 smithi064 systemd[1]: Stopping Ceph mon.a for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:32.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a[42476]: 2024-05-08T22:10:32.458+0000 7f90ea90d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:32.713 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a[42476]: 2024-05-08T22:10:32.458+0000 7f90ea90d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T22:10:33.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 08 22:10:32 smithi064 bash[116905]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-a 2024-05-08T22:10:33.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T22:10:33.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:33.250 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T22:10:33.251 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T22:10:33.251 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.b 2024-05-08T22:10:33.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:33 smithi160 systemd[1]: Stopping Ceph mon.b for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:33.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-b[46939]: 2024-05-08T22:10:33.669+0000 7ff1be2f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:33.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-b[46939]: 2024-05-08T22:10:33.669+0000 7ff1be2f2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T22:10:34.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 22:10:33 smithi160 bash[88485]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-b 2024-05-08T22:10:34.337 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T22:10:34.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:34.391 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T22:10:34.391 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T22:10:34.391 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.c 2024-05-08T22:10:34.795 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:34 smithi186 systemd[1]: Stopping Ceph mon.c for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:34.796 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-c[46596]: 2024-05-08T22:10:34.648+0000 7fc98d5a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:34.796 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-c[46596]: 2024-05-08T22:10:34.648+0000 7fc98d5a5700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T22:10:35.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 08 22:10:34 smithi186 bash[89485]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mon-c 2024-05-08T22:10:35.130 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T22:10:35.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:35.182 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T22:10:35.182 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T22:10:35.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x 2024-05-08T22:10:35.463 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 22:10:35 smithi064 systemd[1]: Stopping Ceph mgr.x for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:35.463 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 22:10:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T22:10:35.407+0000 7f0e0889d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:35.844 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 08 22:10:35 smithi064 bash[117044]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-x 2024-05-08T22:10:35.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T22:10:35.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:35.890 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T22:10:35.890 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T22:10:35.890 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.y 2024-05-08T22:10:36.241 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 22:10:35 smithi160 systemd[1]: Stopping Ceph mgr.y for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:36.544 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 22:10:36 smithi160 bash[88635]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-y 2024-05-08T22:10:36.544 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 22:10:36 smithi160 systemd[1]: ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-08T22:10:36.568 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T22:10:36.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:36.606 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T22:10:36.606 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T22:10:36.606 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.z 2024-05-08T22:10:36.891 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 22:10:36 smithi186 systemd[1]: Stopping Ceph mgr.z for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:37.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 22:10:37 smithi186 bash[89635]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mgr-z 2024-05-08T22:10:37.391 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 08 22:10:37 smithi186 systemd[1]: ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T22:10:37.399 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T22:10:37.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:37.436 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T22:10:37.436 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T22:10:37.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.0 2024-05-08T22:10:37.713 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 22:10:37 smithi064 systemd[1]: Stopping Ceph osd.0 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:37.713 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 22:10:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0[61096]: 2024-05-08T22:10:37.664+0000 7f2f294af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:37.713 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 22:10:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0[61096]: 2024-05-08T22:10:37.664+0000 7f2f294af700 -1 osd.0 84 *** Got signal Terminated *** 2024-05-08T22:10:37.713 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 22:10:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0[61096]: 2024-05-08T22:10:37.664+0000 7f2f294af700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:10:43.713 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 08 22:10:43 smithi064 bash[117186]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-0 2024-05-08T22:10:44.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T22:10:44.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:44.588 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T22:10:44.588 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T22:10:44.588 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.1 2024-05-08T22:10:44.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 22:10:44 smithi160 systemd[1]: Stopping Ceph osd.1 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:45.297 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 22:10:44 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1[51980]: 2024-05-08T22:10:44.900+0000 7fa0f44c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:45.298 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 22:10:44 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1[51980]: 2024-05-08T22:10:44.900+0000 7fa0f44c8700 -1 osd.1 84 *** Got signal Terminated *** 2024-05-08T22:10:45.298 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 22:10:44 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1[51980]: 2024-05-08T22:10:44.900+0000 7fa0f44c8700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:10:51.045 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 22:10:50 smithi160 bash[88775]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-1 2024-05-08T22:10:51.790 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T22:10:51.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:51.832 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T22:10:51.832 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T22:10:51.833 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.2 2024-05-08T22:10:52.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 22:10:51 smithi186 systemd[1]: Stopping Ceph osd.2 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:52.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 22:10:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2[52434]: 2024-05-08T22:10:52.078+0000 7f39d2fcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:52.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 22:10:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2[52434]: 2024-05-08T22:10:52.078+0000 7f39d2fcb700 -1 osd.2 84 *** Got signal Terminated *** 2024-05-08T22:10:52.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 22:10:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2[52434]: 2024-05-08T22:10:52.078+0000 7f39d2fcb700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:10:58.141 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 08 22:10:57 smithi186 bash[89780]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-2 2024-05-08T22:10:59.000 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T22:10:59.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:10:59.041 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T22:10:59.042 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T22:10:59.042 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.3 2024-05-08T22:10:59.463 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 22:10:59 smithi064 systemd[1]: Stopping Ceph osd.3 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:10:59.463 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 22:10:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3[69808]: 2024-05-08T22:10:59.295+0000 7f35c78df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:10:59.463 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 22:10:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3[69808]: 2024-05-08T22:10:59.295+0000 7f35c78df700 -1 osd.3 84 *** Got signal Terminated *** 2024-05-08T22:10:59.463 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 22:10:59 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3[69808]: 2024-05-08T22:10:59.295+0000 7f35c78df700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:04.044 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:03 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:03.729+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:05.044 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:04 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:04.779+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:05.213 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 08 22:11:04 smithi064 bash[117445]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-3 2024-05-08T22:11:06.044 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:05 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:05.752+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:06.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:05 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:05.686+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:06.159 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:05 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:05.717+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:06.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T22:11:06.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:06.208 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T22:11:06.208 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T22:11:06.208 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.4 2024-05-08T22:11:06.532 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 22:11:06 smithi160 systemd[1]: Stopping Ceph osd.4 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:06.795 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:06 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:06.773+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:06.795 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 22:11:06 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4[59914]: 2024-05-08T22:11:06.530+0000 7fa7e83b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:06.795 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 22:11:06 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4[59914]: 2024-05-08T22:11:06.530+0000 7fa7e83b0700 -1 osd.4 84 *** Got signal Terminated *** 2024-05-08T22:11:06.795 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 22:11:06 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4[59914]: 2024-05-08T22:11:06.530+0000 7fa7e83b0700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:07.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:06 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:06.694+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:07.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:06 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:06.765+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:08.044 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:07 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:07.788+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:08.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:07 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:07.658+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:08.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:07 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:07.754+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:09.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:08 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:08.679+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:09.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:08 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:08.794+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:09.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:08 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:08.799+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:09.805 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:09 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:09.525+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:10.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:09 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:09.703+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:10.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:09 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:09.673+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:10.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:09 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:09.804+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:10.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:09 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:09.796+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:10.820 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:10 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:10.496+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:11.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:10 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:10.697+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:11.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:10 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:10.674+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:11.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:10 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:10.819+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:11.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:10 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:10.819+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:11.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:10 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:10.837+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:11.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:11.368+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:11.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:11 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:11.493+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:12.136 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:11 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:11.836+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:12.136 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:11 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:11.836+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:12.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:11.737+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:12.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:11 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:11.669+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:12.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:11 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:11.770+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:12.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:11 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:11.770+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:12.544 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 22:11:12 smithi160 bash[89041]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-4 2024-05-08T22:11:12.634 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:12.328+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:12.812 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:12.471+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:12.812 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:12.471+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:12.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:12.633+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:12.891 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:12.772+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:12.892 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:12 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:12.772+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:13.044 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:12.901+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:13.045 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:12.805+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:13.045 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:12 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:12.805+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:13.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:12.811+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:13.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:12.811+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:13.435 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T22:11:13.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:13.478 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T22:11:13.478 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T22:11:13.478 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.5 2024-05-08T22:11:13.587 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:13 smithi186 systemd[1]: Stopping Ceph osd.5 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:13.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:13.330+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:13.791 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:13.464+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:13.791 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:13.464+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:13.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:13.586+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:13.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:13.586+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:13.891 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:13.726+0000 7fc597b9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:13.892 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:13.726+0000 7fc597b9f700 -1 osd.5 84 *** Got signal Terminated *** 2024-05-08T22:11:13.892 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:13.726+0000 7fc597b9f700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:13.892 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:13.773+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:13.892 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:13 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:13.773+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:14.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:13.790+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:14.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:13 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:13.790+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:14.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:13.867+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:14.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:13.850+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:14.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:13 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:13.850+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:14.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:14.317+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:14.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:14.553+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:14.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:14.553+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:14.789 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:14.512+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:14.789 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:14.512+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:15.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:14.798+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:15.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:14 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:14.798+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:15.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:14.788+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:15.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:14 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:14.788+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:15.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:14 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:14.850+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:15.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:14 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:14.863+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:15.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:14 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:14.863+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:15.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:15.359+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:15.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:15.526+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:15.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:15.526+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:15.797 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:15.498+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:15.798 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:15.498+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:16.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:15.783+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:16.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:15 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:15.783+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:16.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:15.797+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:16.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:15 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:15.797+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:16.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:15 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:15.849+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:16.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:15 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:15.901+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:16.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:15 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:15.901+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:16.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:16.368+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:16.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:16.575+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:16.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:16.575+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:16.808 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:16.526+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:16.809 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:16.526+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:17.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:16.744+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:17.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:16 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:16.744+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:17.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:16.807+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:17.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:16 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:16.808+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:17.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:16 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:16.878+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:17.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:16 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:16.857+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:17.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:16 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:16.857+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:17.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:17 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:17.330+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:17.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:17 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:17.534+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:17.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:17 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:17.534+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:17.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:17.561+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:17.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:17.561+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:17.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:17.783+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:17.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:17.783+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:18.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:17 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:17.714+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:18.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:17 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:17.714+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:18.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:17.899+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:18.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:17.834+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:18.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:17.834+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:18.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:17 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:17.834+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:18.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:18 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:18.357+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:18.643 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:18 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:18.543+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:18.643 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:18 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:18.543+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:18.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:18.570+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:18.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:18.570+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:18.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:18.570+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:18.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:18.744+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:18.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:18 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:18.744+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:19.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:18 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:18.666+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.524842+0000 front 2024-05-08T22:10:41.524650+0000 (oldest deadline 2024-05-08T22:11:05.024615+0000) 2024-05-08T22:11:19.141 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:18 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5[60428]: 2024-05-08T22:11:18.666+0000 7fc59098e700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.925849+0000 front 2024-05-08T22:10:47.925976+0000 (oldest deadline 2024-05-08T22:11:12.626033+0000) 2024-05-08T22:11:19.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:18.905+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:19.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:18.905+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:19.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:18.862+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:19.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:18.862+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:19.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:18 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:18.862+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:19.641 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 08 22:11:19 smithi186 bash[90043]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-5 2024-05-08T22:11:19.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:19 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:19.373+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:19.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:19 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:19.373+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:19.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:19 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:19.523+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:19.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:19 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:19.523+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:19.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:19.613+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:19.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:19.613+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:19.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:19.613+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:19.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:19.727+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:19.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:19.727+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:19.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:19 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:19.728+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:20.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:19 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:19.950+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:20.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:19 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:19.950+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:20.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:19 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:19.823+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:20.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:19 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:19.823+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:20.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:19 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:19.823+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:20.581 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:20 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:20.328+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:20.581 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:20 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:20.328+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:20.581 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:20 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:20.525+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:20.581 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:20 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:20.525+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:20.581 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:20 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:20.525+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:20.781 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T22:11:20.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:20.822 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T22:11:20.822 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T22:11:20.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.6 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:20.697+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:20.697+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:20.697+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:20.630+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:20.630+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:20 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:20.630+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:20.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:20 smithi064 systemd[1]: Stopping Ceph osd.6 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:21.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:20 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:20.916+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:21.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:20 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:20.916+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:21.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:20 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:20.821+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:21.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:20 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:20.821+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:21.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:20 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:20.821+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:21.463 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:21.148+0000 7f044302d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:21.463 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:21.148+0000 7f044302d700 -1 osd.6 84 *** Got signal Terminated *** 2024-05-08T22:11:21.463 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:21.148+0000 7f044302d700 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:21.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:21 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:21.289+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:21.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:21 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:21.289+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:21.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:21 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:21.516+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:21.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:21 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:21.516+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:21.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:21 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:21.516+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:21.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:21.670+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:21.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:21.670+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:21.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:21.670+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:21.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:21.651+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:21.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:21.651+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:21.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:21 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:21.651+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:22.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:21.912+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:22.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:21.912+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:22.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:21.866+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:22.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:21.866+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:22.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:21.866+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:22.519 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:22 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:22.251+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:22.519 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:22 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:22.251+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:22.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:22 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:22.518+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:22.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:22 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:22.518+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:22.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:22 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:22.518+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:22.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:22.647+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:22.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:22.647+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:22.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:22.647+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:22.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:22.684+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:22.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:22.684+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:22.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:22 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:22.684+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:23.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:22 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:22.906+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:23.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:22 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:22.906+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:23.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:22 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:22.884+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:23.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:22 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:22.884+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:23.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:22 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:22.884+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:23.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:23 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:23.480+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:23.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:23 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:23.480+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:23.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:23 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:23.480+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:23.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:23 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:23.247+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:23.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:23 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:23.247+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:23.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:23 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:23.601+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:23.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:23 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:23.601+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:23.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:23 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:23.601+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:23.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:23 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:23.661+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:23.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:23 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:23.661+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:23.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:23 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:23.661+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:24.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:23 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:23.911+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:24.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:23 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:23.911+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:24.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:23 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:23.919+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:24.296 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:23 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:23.919+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:24.296 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:23 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:23.919+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:24.489 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:24 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:24.213+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:24.490 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:24 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:24.213+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:24.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:24 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:24.489+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:24.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:24 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:24.489+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:24.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:24 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:24.489+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:24.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:24 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:24.644+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:24.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:24 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:24.644+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:24.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:24 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:24.644+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:24.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:24 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:24.678+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:24.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:24 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:24.678+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:24.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:24 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:24.678+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:25.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:24 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:24.951+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:25.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:24 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:24.951+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:25.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:24 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:24.957+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:25.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:24 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:24.957+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:25.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:24 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:24.957+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:25.496 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:25 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:25.187+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:25.496 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:25 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:25.187+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:25.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:25 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:25.495+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:25.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:25 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:25.495+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:25.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:25 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:25.495+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:25.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:25.613+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:39.534864+0000 front 2024-05-08T22:10:39.534884+0000 (oldest deadline 2024-05-08T22:11:08.935802+0000) 2024-05-08T22:11:25.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:25.613+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.936262+0000 front 2024-05-08T22:10:48.936344+0000 (oldest deadline 2024-05-08T22:11:12.436294+0000) 2024-05-08T22:11:25.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6[77957]: 2024-05-08T22:11:25.613+0000 7f043fe24700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.436802+0000 front 2024-05-08T22:10:52.436937+0000 (oldest deadline 2024-05-08T22:11:17.736828+0000) 2024-05-08T22:11:25.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:25.647+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:25.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:25.647+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:25.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:25 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:25.647+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:26.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:25.958+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:26.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:25.958+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:26.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:25.958+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:26.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:25.974+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:26.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:25.974+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:26.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:25.974+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:26.516 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:26 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:26.173+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:26.516 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:26 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:26.173+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:26.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:26 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:26.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:26.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:26 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:26.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:26.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:26 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:26.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:26.963 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 08 22:11:26 smithi064 bash[117708]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-6 2024-05-08T22:11:26.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:26 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:26.604+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:26.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:26 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:26.604+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:26.964 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:26 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:26.604+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:27.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:27.009+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:27.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:27.009+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:27.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:27.009+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:27.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:26 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:26.962+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:27.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:26 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:26.962+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:27.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:26 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:26.962+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:27.489 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:27 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:27.184+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:27.489 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:27 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:27.184+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:27.881 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:27 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:27.606+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:27.881 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:27 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:27.606+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:27.881 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:27 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:27.606+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:27.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T22:11:27.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:27 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:27.488+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:27.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:27 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:27.488+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:27.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:27 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:27.488+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:27.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:27.931 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T22:11:27.931 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T22:11:27.931 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.7 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:27.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:27.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:27.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:27 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:27.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:27 smithi160 systemd[1]: Stopping Ceph osd.7 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:28.038+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:28.038+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:28.038+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:28.177+0000 7f5baf27d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:28.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:28.177+0000 7f5baf27d700 -1 osd.7 84 *** Got signal Terminated *** 2024-05-08T22:11:28.296 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:28.177+0000 7f5baf27d700 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:28.527 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:28.210+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:28.527 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:28.210+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:28.527 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:28.210+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:28.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:28.526+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:28.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:28.526+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:28.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:28 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:28.526+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:28.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:28.636+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:28.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:28.636+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:28.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:28.636+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:28.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:28 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:28.637+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:29.076+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:29.076+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:29.076+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:28.960+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:28.960+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:28.960+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:29.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:28 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:28.960+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:29.499 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:29 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:29.239+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:29.500 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:29 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:29.239+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:29.500 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:29 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:29.239+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:29.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:29 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:29.499+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:29.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:29 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:29.499+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:29.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:29 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:29.499+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:29.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:29.630+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:29.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:29.630+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:29.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:29.630+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:29.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:29 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:29.630+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:30.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:30.127+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:30.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:30.127+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:30.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:30.127+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:30.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:29.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:30.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:29.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:30.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:29.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:30.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:29.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:30.510 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:30 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:30.200+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:30.511 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:30 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:30.200+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:30.511 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:30 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:30.200+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:30.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:30 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:30.510+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:30.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:30 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:30.510+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:30.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:30 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:30.510+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:30.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:30 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:30.668+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:30.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:30 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:30.668+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:30.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:30 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:30.668+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:30.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:30 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:30.668+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:31.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:31.123+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:31.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:31.123+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:31.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:31.123+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:31.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:30.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:31.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:30.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:31.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:30.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:31.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:30 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:30.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:31.536 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:31 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:31.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:31.536 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:31 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:31.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:31.536 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:31 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:31.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:31.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:31 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:31.536+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:31.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:31 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:31.536+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:31.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:31 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:31.536+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:31.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:31 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:31.709+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:31.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:31 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:31.709+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:31.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:31 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:31.709+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:31.963 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:31 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:31.709+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:32.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:32.097+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:32.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:32.097+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:32.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:32.097+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:32.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:31.908+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:32.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:31.908+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:32.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:31.908+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:32.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:31 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:31.908+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:32.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:32 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:32.489+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:32.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:32 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:32.489+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:32.641 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:32 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:32.489+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:32.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:32 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:32.270+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:32.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:32 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:32.270+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:32.642 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:32 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:32.270+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:33.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:32.726+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:33.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:32.726+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:33.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:32.726+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:33.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:32 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:32.726+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:33.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:33.058+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.487710+0000 front 2024-05-08T22:10:47.487614+0000 (oldest deadline 2024-05-08T22:11:12.187714+0000) 2024-05-08T22:11:33.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:33.058+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.188162+0000 front 2024-05-08T22:10:52.188209+0000 (oldest deadline 2024-05-08T22:11:17.988804+0000) 2024-05-08T22:11:33.295 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7[67893]: 2024-05-08T22:11:33.058+0000 7f5bac074700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.591194+0000 front 2024-05-08T22:11:03.591271+0000 (oldest deadline 2024-05-08T22:11:25.291254+0000) 2024-05-08T22:11:33.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:32.875+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:33.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:32.875+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:33.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:32.875+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:33.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:32 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:32.875+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:33.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:33.285+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:33.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:33.285+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:33.641 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:33.285+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:33.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:33.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:33.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:33.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:33.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:33.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:33.642 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:33 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:33.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:34.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:33 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:33.772+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:34.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:33 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:33.772+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:34.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:33 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:33.772+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:34.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:33 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:33.772+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:34.294 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 22:11:33 smithi160 bash[89297]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-7 2024-05-08T22:11:34.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:33.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:34.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:33.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:34.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:33.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:34.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:33.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:34.555 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:34.255+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:34.556 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:34.255+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:34.556 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:34.255+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:34.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:34.555+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:34.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:34.555+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:34.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:34.555+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:34.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:34 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:34.555+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:35.158 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:34 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:34.933+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:35.158 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:34 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:34.933+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:35.158 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:34 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:34.933+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:35.158 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:34 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:34.933+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:35.165 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T22:11:35.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:35.206 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T22:11:35.206 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T22:11:35.206 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.8 2024-05-08T22:11:35.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:34.774+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:35.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:34.774+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:35.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:34.774+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:35.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:34 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:34.774+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:35.528 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 systemd[1]: Stopping Ceph osd.8 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:35.528 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.294+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:35.529 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.294+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:35.529 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.294+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:35.529 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.294+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.059393+0000 front 2024-05-08T22:11:10.059438+0000 (oldest deadline 2024-05-08T22:11:34.759313+0000) 2024-05-08T22:11:35.529 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.447+0000 7f0e80928700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:35.529 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.447+0000 7f0e80928700 -1 osd.8 84 *** Got signal Terminated *** 2024-05-08T22:11:35.529 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:35.447+0000 7f0e80928700 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:35.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:35.527+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:35.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:35.527+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:35.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:35.527+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:35.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:35 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:35.527+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:36.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:35.812+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:36.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:35.812+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:36.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:35.812+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:36.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:35 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:35.812+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:36.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:35 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:35.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:36.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:35 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:35.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:36.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:35 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:35.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:36.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:35 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:35.938+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:36.530 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:36.248+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:36.530 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:36.248+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:36.531 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:36.248+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:36.531 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:36.248+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.059393+0000 front 2024-05-08T22:11:10.059438+0000 (oldest deadline 2024-05-08T22:11:34.759313+0000) 2024-05-08T22:11:36.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:36.529+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:36.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:36.529+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:36.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:36.529+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:36.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:36 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:36.529+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:37.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:36.809+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:37.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:36.809+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:37.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:36.809+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:37.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:36 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:36.809+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:37.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:36 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:36.889+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:37.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:36 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:36.889+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:37.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:36 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:36.889+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:37.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:36 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:36.889+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:37.562 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:37.271+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:37.562 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:37.271+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:37.562 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:37.271+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:37.562 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:37.271+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.059393+0000 front 2024-05-08T22:11:10.059438+0000 (oldest deadline 2024-05-08T22:11:34.759313+0000) 2024-05-08T22:11:37.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:37.561+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:37.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:37.561+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:37.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:37.561+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:37.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:37.561+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:38.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:37.815+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:38.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:37.815+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:38.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:37.815+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:38.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:37 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:37.815+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:38.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:37 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:37.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:38.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:37 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:37.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:38.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:37 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:37.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:38.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:37 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:37.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:38.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:37 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:37.895+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:38.534 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:38.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:38.534 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:38.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:38.534 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:38.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:38.535 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:38.232+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.059393+0000 front 2024-05-08T22:11:10.059438+0000 (oldest deadline 2024-05-08T22:11:34.759313+0000) 2024-05-08T22:11:38.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:38.533+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:38.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:38.533+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:38.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:38.533+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:38.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:38 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:38.533+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:39.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:38.850+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:39.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:38.850+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:39.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:38.850+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:39.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:38 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:38.850+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:39.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:38 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:38.911+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:39.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:38 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:38.911+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:39.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:38 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:38.911+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:39.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:38 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:38.911+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:39.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:38 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:38.911+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:39.573 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:39.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:39.574 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:39.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:39.574 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:39.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:39.574 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:39.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.059393+0000 front 2024-05-08T22:11:10.059438+0000 (oldest deadline 2024-05-08T22:11:34.759313+0000) 2024-05-08T22:11:39.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:39.573+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:39.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:39.573+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:39.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:39.573+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:39.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:39 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:39.573+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:40.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:39.869+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:40.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:39.869+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:40.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:39.869+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:40.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:39.869+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:40.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:39 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:39.869+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:40.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:39 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:39.893+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:40.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:39 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:39.893+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:40.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:39 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:39.893+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:40.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:39 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:39.893+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:40.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:39 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:39.893+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:40.601 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:40.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:45.054866+0000 front 2024-05-08T22:10:45.054892+0000 (oldest deadline 2024-05-08T22:11:10.955042+0000) 2024-05-08T22:11:40.601 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:40.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:56.256143+0000 front 2024-05-08T22:10:56.256159+0000 (oldest deadline 2024-05-08T22:11:19.156276+0000) 2024-05-08T22:11:40.601 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:40.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.058463+0000 front 2024-05-08T22:11:03.058551+0000 (oldest deadline 2024-05-08T22:11:27.758291+0000) 2024-05-08T22:11:40.601 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:40.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.059393+0000 front 2024-05-08T22:11:10.059438+0000 (oldest deadline 2024-05-08T22:11:34.759313+0000) 2024-05-08T22:11:40.601 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8[68420]: 2024-05-08T22:11:40.224+0000 7f0e7d71f700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.759787+0000 front 2024-05-08T22:11:14.759801+0000 (oldest deadline 2024-05-08T22:11:39.960463+0000) 2024-05-08T22:11:40.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:40.600+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:40.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:40.600+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:40.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:40.600+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:40.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:40 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:40.600+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:41.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:40 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:40.865+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:41.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:40 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:40.865+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:41.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:40 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:40.865+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:41.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:40 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:40.865+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:41.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:40 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:40.865+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:41.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:40 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:40.939+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:41.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:40 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:40.939+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:41.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:40 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:40.939+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:41.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:40 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:40.939+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:41.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:40 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:40.939+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:41.609 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 08 22:11:41 smithi186 bash[90298]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-8 2024-05-08T22:11:41.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:41 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:41.608+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:41.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:41 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:41.608+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:41.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:41 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:41.608+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:41.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:41 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:41.608+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:42.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:41.826+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:42.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:41.826+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:42.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:41.826+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:42.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:41.826+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:42.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:41 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:41.826+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:42.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:41.958+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:42.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:41.958+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:42.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:41.958+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:42.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:41.958+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:42.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:41 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:41.958+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:42.411 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T22:11:42.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:42.452 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T22:11:42.452 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T22:11:42.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.9 2024-05-08T22:11:42.845 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 systemd[1]: Stopping Ceph osd.9 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:42.845 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.698+0000 7f7eced0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:42.845 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.698+0000 7f7eced0c700 -1 osd.9 84 *** Got signal Terminated *** 2024-05-08T22:11:42.845 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.698+0000 7f7eced0c700 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:42.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:42 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:42.643+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:42.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:42 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:42.643+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:42.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:42 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:42.643+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:42.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:42 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:42.643+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:42.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:42 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:42.643+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:43.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.844+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:43.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.844+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:43.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.844+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:43.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.844+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:43.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:42 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:42.844+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:43.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:42 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:42.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:43.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:42 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:42.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:43.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:42 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:42.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:43.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:42 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:42.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:43.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:42 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:42.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:43.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:42 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:42.950+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:43.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:43 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:43.633+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:43.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:43 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:43.633+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:43.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:43 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:43.633+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:43.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:43 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:43.633+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:43.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:43 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:43.633+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:44.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:43 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:43.848+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:44.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:43 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:43.848+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:44.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:43 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:43.848+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:44.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:43 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:43.848+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:44.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:43 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:43.848+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:44.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:43 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:43.982+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:44.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:43 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:43.982+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:44.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:43 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:43.982+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:44.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:43 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:43.982+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:44.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:43 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:43.982+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:44.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:43 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:43.982+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:44.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:44 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:44.611+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:44.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:44 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:44.611+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:44.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:44 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:44.611+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:44.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:44 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:44.611+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:44.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:44 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:44.611+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:45.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:44 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:44.838+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:45.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:44 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:44.838+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:45.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:44 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:44.838+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:45.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:44 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:44.838+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:45.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:44 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:44.838+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:45.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:45 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:45.004+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:45.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:45 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:45.004+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:45.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:45 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:45.004+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:45.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:45 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:45.004+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:45.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:45 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:45.004+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:45.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:45 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:45.004+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:45.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:45.638+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:45.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:45.638+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:45.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:45.638+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:45.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:45.638+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:45.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:45.638+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:46.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:45 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:45.878+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:46.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:45 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:45.878+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:46.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:45 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:45.878+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:46.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:45 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:45.878+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:46.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:45 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:45.878+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:46.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:46 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:46.046+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:46.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:46 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:46.046+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:46.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:46 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:46.046+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:46.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:46 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:46.046+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:46.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:46 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:46.046+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:46.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:46 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:46.046+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:46.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:46 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:46.598+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:46.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:46 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:46.598+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:46.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:46 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:46.598+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:46.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:46 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:46.598+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:46.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:46 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:46.598+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:46 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:46.904+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:42.590048+0000 front 2024-05-08T22:10:42.590075+0000 (oldest deadline 2024-05-08T22:11:04.890276+0000) 2024-05-08T22:11:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:46 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:46.904+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:47.191708+0000 front 2024-05-08T22:10:47.191838+0000 (oldest deadline 2024-05-08T22:11:10.691781+0000) 2024-05-08T22:11:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:46 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:46.904+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.392867+0000 front 2024-05-08T22:10:52.392910+0000 (oldest deadline 2024-05-08T22:11:18.793677+0000) 2024-05-08T22:11:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:46 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:46.904+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.995008+0000 front 2024-05-08T22:11:03.994912+0000 (oldest deadline 2024-05-08T22:11:28.095029+0000) 2024-05-08T22:11:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:46 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9[86774]: 2024-05-08T22:11:46.904+0000 7f7ecbb03700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:14.496812+0000 front 2024-05-08T22:11:14.496674+0000 (oldest deadline 2024-05-08T22:11:39.796550+0000) 2024-05-08T22:11:47.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:47.005+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:47.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:47.005+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:47.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:47.005+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:47.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:47.005+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:47.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:47.005+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:47.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:47 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:47.005+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:47.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:47.584+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:47.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:47.584+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:47.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:47.584+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:47.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:47.584+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:47.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:47 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:47.584+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:48.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:48 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:48.042+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:48.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:48 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:48.042+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:48.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:48 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:48.042+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:48.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:48 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:48.042+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:48.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:48 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:48.042+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:48.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:48 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:48.042+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:48.713 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 08 22:11:48 smithi064 bash[117965]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-9 2024-05-08T22:11:48.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:48 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:48.539+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:48.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:48 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:48.539+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:48.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:48 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:48.539+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:48.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:48 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:48.539+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:48.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:48 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:48.539+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:49.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.021+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:49.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.021+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:49.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.021+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:49.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.021+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:49.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.021+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:49.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.021+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:49.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T22:11:49.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:49.662 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T22:11:49.662 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T22:11:49.662 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.10 2024-05-08T22:11:49.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:49.550+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:49.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:49.550+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:49.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:49.550+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:49.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:49.550+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:49.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:49.550+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:49.991 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 systemd[1]: Stopping Ceph osd.10 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:50.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.990+0000 7f2d4bf98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:50.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.990+0000 7f2d4bf98700 -1 osd.10 84 *** Got signal Terminated *** 2024-05-08T22:11:50.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:49 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:49.990+0000 7f2d4bf98700 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:50.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:50 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:50.069+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.628669+0000 front 2024-05-08T22:11:24.628704+0000 (oldest deadline 2024-05-08T22:11:49.928444+0000) 2024-05-08T22:11:50.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:50.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:50.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:50.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:50.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:50.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:50.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:50.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:50.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:50.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:50.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:50 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:50.515+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:51.294 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:51.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:51.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:51.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:51.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:51.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:51.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:51 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:51.036+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.628669+0000 front 2024-05-08T22:11:24.628704+0000 (oldest deadline 2024-05-08T22:11:49.928444+0000) 2024-05-08T22:11:51.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:51.522+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:51.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:51.522+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:51.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:51.522+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:51.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:51.522+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:51.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:51.522+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:51.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:51 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:51.522+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:52.544 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:52.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:52.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:52.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:52.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:52.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:52.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:52 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:52.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.628669+0000 front 2024-05-08T22:11:24.628704+0000 (oldest deadline 2024-05-08T22:11:49.928444+0000) 2024-05-08T22:11:52.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:52.549+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:52.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:52.549+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:52.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:52.549+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:52.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:52.549+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:52.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:52.549+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:52.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:52 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:52.549+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:53.544 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:53.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:53.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:53.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:53.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:53.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:53.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:53 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:53.098+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.628669+0000 front 2024-05-08T22:11:24.628704+0000 (oldest deadline 2024-05-08T22:11:49.928444+0000) 2024-05-08T22:11:53.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:53 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:53.538+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:53.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:53 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:53.538+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:53.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:53 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:53.538+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:53.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:53 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:53.538+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:53.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:53 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:53.538+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:53.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:53 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:53.538+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:54.544 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:38.521551+0000 front 2024-05-08T22:10:38.521535+0000 (oldest deadline 2024-05-08T22:11:03.221613+0000) 2024-05-08T22:11:54.544 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:49.623296+0000 front 2024-05-08T22:10:49.623273+0000 (oldest deadline 2024-05-08T22:11:11.823956+0000) 2024-05-08T22:11:54.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:52.324924+0000 front 2024-05-08T22:10:52.325026+0000 (oldest deadline 2024-05-08T22:11:17.624968+0000) 2024-05-08T22:11:54.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-08T22:11:03.525988+0000 front 2024-05-08T22:11:03.525947+0000 (oldest deadline 2024-05-08T22:11:27.625938+0000) 2024-05-08T22:11:54.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:07.626268+0000 front 2024-05-08T22:11:07.626295+0000 (oldest deadline 2024-05-08T22:11:37.026859+0000) 2024-05-08T22:11:54.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:17.027431+0000 front 2024-05-08T22:11:17.027371+0000 (oldest deadline 2024-05-08T22:11:42.927364+0000) 2024-05-08T22:11:54.545 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:54 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10[76693]: 2024-05-08T22:11:54.054+0000 7f2d44d87700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.628669+0000 front 2024-05-08T22:11:24.628704+0000 (oldest deadline 2024-05-08T22:11:49.928444+0000) 2024-05-08T22:11:54.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:54 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:54.540+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:54.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:54 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:54.540+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:54.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:54 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:54.540+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:54.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:54 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:54.540+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:54.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:54 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:54.540+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:54.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:54 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:54.540+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:55.794 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 22:11:55 smithi160 bash[89553]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-10 2024-05-08T22:11:55.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:55.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:55.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:55.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:55.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:55.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:55.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:55 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:55.577+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:11:56.703 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T22:11:56.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:11:56.745 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T22:11:56.745 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T22:11:56.745 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.11 2024-05-08T22:11:56.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.568+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:11:56.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 systemd[1]: Stopping Ceph osd.11 for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:11:57.391 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.985+0000 7f7efe6e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:11:57.391 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.985+0000 7f7efe6e2700 -1 osd.11 84 *** Got signal Terminated *** 2024-05-08T22:11:57.391 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:56 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:56.985+0000 7f7efe6e2700 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:11:57.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:57.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:57.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:57.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:57.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:57.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:57.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:57 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:57.603+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:11:58.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:11:58.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:11:58.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:11:58.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:11:58.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:11:58.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:11:58.892 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:58 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:58.636+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:12:00.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:12:00.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:12:00.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:12:00.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:12:00.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:12:00.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:12:00.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:11:59 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:11:59.686+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:12:01.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:12:01.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:12:01.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:12:01.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:12:01.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:12:01.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:12:01.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:00 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:00.681+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:12:02.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-08T22:10:41.136789+0000 front 2024-05-08T22:10:41.136622+0000 (oldest deadline 2024-05-08T22:11:08.737253+0000) 2024-05-08T22:12:02.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-08T22:10:48.737822+0000 front 2024-05-08T22:10:48.737984+0000 (oldest deadline 2024-05-08T22:11:13.437803+0000) 2024-05-08T22:12:02.141 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-08T22:10:53.938657+0000 front 2024-05-08T22:10:53.938778+0000 (oldest deadline 2024-05-08T22:11:19.838795+0000) 2024-05-08T22:12:02.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-08T22:11:10.340918+0000 front 2024-05-08T22:11:10.340995+0000 (oldest deadline 2024-05-08T22:11:33.240894+0000) 2024-05-08T22:12:02.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-08T22:11:16.141893+0000 front 2024-05-08T22:11:16.141944+0000 (oldest deadline 2024-05-08T22:11:42.042096+0000) 2024-05-08T22:12:02.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-08T22:11:24.343515+0000 front 2024-05-08T22:11:24.343592+0000 (oldest deadline 2024-05-08T22:11:49.643355+0000) 2024-05-08T22:12:02.142 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:01 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11[77237]: 2024-05-08T22:12:01.692+0000 7f7ef74d1700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-08T22:11:29.643906+0000 front 2024-05-08T22:11:29.643972+0000 (oldest deadline 2024-05-08T22:11:55.543781+0000) 2024-05-08T22:12:02.891 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 08 22:12:02 smithi186 bash[90553]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-osd-11 2024-05-08T22:12:03.916 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T22:12:03.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:03.956 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T22:12:03.957 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T22:12:03.957 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.a 2024-05-08T22:12:04.463 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:12:03 smithi064 systemd[1]: Stopping Ceph mds.a for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:04.463 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:12:04 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a[94879]: 2024-05-08T22:12:04.203+0000 7fbc3d5dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:04.463 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:12:04 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a[94879]: 2024-05-08T22:12:04.203+0000 7fbc3d5dd700 -1 mds.a *** got signal Terminated *** 2024-05-08T22:12:08.673 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 08 22:12:08 smithi064 bash[118220]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-a 2024-05-08T22:12:08.679 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T22:12:08.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:08.715 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T22:12:08.716 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T22:12:08.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.d 2024-05-08T22:12:08.963 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 22:12:08 smithi064 systemd[1]: Stopping Ceph mds.d for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:09.463 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 22:12:09 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-d[93963]: 2024-05-08T22:12:09.010+0000 7fab69da3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:09.463 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 22:12:09 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-d[93963]: 2024-05-08T22:12:09.010+0000 7fab69da3700 -1 mds.d *** got signal Terminated *** 2024-05-08T22:12:11.861 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 08 22:12:11 smithi064 bash[118361]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-d 2024-05-08T22:12:11.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T22:12:11.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:11.904 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T22:12:11.904 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T22:12:11.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.g 2024-05-08T22:12:12.199 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:12:11 smithi064 systemd[1]: Stopping Ceph mds.g for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:12.463 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:12:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g[94493]: 2024-05-08T22:12:12.198+0000 7f1c67cf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:12.463 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:12:12 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g[94493]: 2024-05-08T22:12:12.198+0000 7f1c67cf0700 -1 mds.g *** got signal Terminated *** 2024-05-08T22:12:16.704 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 08 22:12:16 smithi064 bash[118499]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-g 2024-05-08T22:12:16.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T22:12:16.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:16.748 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T22:12:16.748 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T22:12:16.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.j 2024-05-08T22:12:16.963 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 22:12:16 smithi064 systemd[1]: Stopping Ceph mds.j for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:17.463 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 22:12:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-j[93578]: 2024-05-08T22:12:17.010+0000 7fbaac1b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:17.463 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 22:12:17 smithi064 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-j[93578]: 2024-05-08T22:12:17.010+0000 7fbaac1b9700 -1 mds.j *** got signal Terminated *** 2024-05-08T22:12:21.444 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 08 22:12:21 smithi064 bash[118648]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-j 2024-05-08T22:12:21.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T22:12:21.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:21.487 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T22:12:21.487 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T22:12:21.487 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.b 2024-05-08T22:12:21.794 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:12:21 smithi160 systemd[1]: Stopping Ceph mds.b for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:21.795 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:12:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: 2024-05-08T22:12:21.727+0000 7fa08cfb1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:21.795 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:12:21 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b[82968]: 2024-05-08T22:12:21.727+0000 7fa08cfb1700 -1 mds.b *** got signal Terminated *** 2024-05-08T22:12:25.228 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 22:12:24 smithi160 bash[89815]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-b 2024-05-08T22:12:25.235 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T22:12:25.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:25.272 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T22:12:25.273 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T22:12:25.273 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.e 2024-05-08T22:12:25.536 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:12:25 smithi160 systemd[1]: Stopping Ceph mds.e for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:25.794 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:12:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e[82283]: 2024-05-08T22:12:25.534+0000 7ff5df460700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:25.835 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:12:25 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e[82283]: 2024-05-08T22:12:25.534+0000 7ff5df460700 -1 mds.e *** got signal Terminated *** 2024-05-08T22:12:29.234 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 22:12:28 smithi160 bash[89953]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-e 2024-05-08T22:12:29.239 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T22:12:29.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:29.273 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T22:12:29.273 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T22:12:29.273 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.h 2024-05-08T22:12:29.535 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:12:29 smithi160 systemd[1]: Stopping Ceph mds.h for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:29.794 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:12:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h[82623]: 2024-05-08T22:12:29.534+0000 7ff6316e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:29.795 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:12:29 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h[82623]: 2024-05-08T22:12:29.534+0000 7ff6316e3700 -1 mds.h *** got signal Terminated *** 2024-05-08T22:12:33.231 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 22:12:32 smithi160 bash[90090]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-h 2024-05-08T22:12:33.237 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T22:12:33.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:33.270 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T22:12:33.270 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T22:12:33.270 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.k 2024-05-08T22:12:33.534 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:12:33 smithi160 systemd[1]: Stopping Ceph mds.k for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:33.794 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:12:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k[81932]: 2024-05-08T22:12:33.534+0000 7fdb7c091700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:33.794 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:12:33 smithi160 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k[81932]: 2024-05-08T22:12:33.534+0000 7fdb7c091700 -1 mds.k *** got signal Terminated *** 2024-05-08T22:12:37.277 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 22:12:36 smithi160 bash[90229]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-k 2024-05-08T22:12:37.284 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T22:12:37.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:37.319 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T22:12:37.319 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T22:12:37.320 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.c 2024-05-08T22:12:37.641 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:12:37 smithi186 systemd[1]: Stopping Ceph mds.c for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:37.642 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:12:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c[83335]: 2024-05-08T22:12:37.563+0000 7fdf9c8ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:37.642 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:12:37 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c[83335]: 2024-05-08T22:12:37.563+0000 7fdf9c8ea700 -1 mds.c *** got signal Terminated *** 2024-05-08T22:12:41.219 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 08 22:12:40 smithi186 bash[90815]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-c 2024-05-08T22:12:41.225 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T22:12:41.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:41.264 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T22:12:41.264 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T22:12:41.264 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.f 2024-05-08T22:12:41.531 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:12:41 smithi186 systemd[1]: Stopping Ceph mds.f for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:41.891 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:12:41 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f[82984]: 2024-05-08T22:12:41.529+0000 7fa86c456700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:41.891 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:12:41 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f[82984]: 2024-05-08T22:12:41.529+0000 7fa86c456700 -1 mds.f *** got signal Terminated *** 2024-05-08T22:12:45.188 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 08 22:12:44 smithi186 bash[90953]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-f 2024-05-08T22:12:45.194 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T22:12:45.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:45.229 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T22:12:45.229 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T22:12:45.229 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.i 2024-05-08T22:12:45.480 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:12:45 smithi186 systemd[1]: Stopping Ceph mds.i for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:45.891 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:12:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i[83678]: 2024-05-08T22:12:45.478+0000 7fa2b47cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:45.891 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:12:45 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i[83678]: 2024-05-08T22:12:45.478+0000 7fa2b47cb700 -1 mds.i *** got signal Terminated *** 2024-05-08T22:12:49.141 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 08 22:12:48 smithi186 bash[91091]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-i 2024-05-08T22:12:49.276 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T22:12:49.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:49.309 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T22:12:49.310 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T22:12:49.310 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.l 2024-05-08T22:12:49.571 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:12:49 smithi186 systemd[1]: Stopping Ceph mds.l for 71e641de-0d84-11ef-bc97-c7b262605968... 2024-05-08T22:12:49.891 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:12:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l[82639]: 2024-05-08T22:12:49.569+0000 7f984484f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:12:49.891 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:12:49 smithi186 ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l[82639]: 2024-05-08T22:12:49.569+0000 7f984484f700 -1 mds.l *** got signal Terminated *** 2024-05-08T22:12:53.141 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 08 22:12:52 smithi186 bash[91231]: ceph-71e641de-0d84-11ef-bc97-c7b262605968-mds-l 2024-05-08T22:12:53.215 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-71e641de-0d84-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T22:12:53.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:12:53.250 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T22:12:53.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:12:53.442 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T22:12:54.925 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:12:55.112 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T22:12:56.518 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:12:56.706 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T22:12:58.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:12:58.148 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:12:58.184 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:12:58.214 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T22:12:58.215 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi064/crash 2024-05-08T22:12:58.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:12:58.329 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:12:58.329 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:12:58.331 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi160/crash 2024-05-08T22:12:58.332 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:12:58.377 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:12:58.377 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:12:58.379 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi186/crash 2024-05-08T22:12:58.380 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:12:58.422 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/71e641de-0d84-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:12:58.422 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:12:58.423 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T22:12:58.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/71e641de-0d84-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-05-08T22:12:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:grep: /var/log/ceph/71e641de-0d84-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-08T22:12:58.464 INFO:tasks.cephadm:Compressing logs... 2024-05-08T22:12:58.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:12:58.507 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:12:58.509 DEBUG:teuthology.orchestra.run.smithi186:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:12:58.535 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:12:58.536 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:12:58.539 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:14:44.376 INFO:tasks.cephadm:Archiving logs... 2024-05-08T22:14:44.378 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi064/log 2024-05-08T22:14:44.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:14:56.534 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi160/log 2024-05-08T22:14:56.535 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:15:06.708 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi186/log 2024-05-08T22:15:06.709 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:15:19.189 INFO:tasks.cephadm:Removing cluster... 2024-05-08T22:15:19.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --force 2024-05-08T22:15:19.397 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T22:15:19.764 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --force 2024-05-08T22:15:19.961 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T22:15:20.278 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 71e641de-0d84-11ef-bc97-c7b262605968 --force 2024-05-08T22:15:20.479 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 71e641de-0d84-11ef-bc97-c7b262605968 2024-05-08T22:15:20.822 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T22:15:20.822 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:15:20.844 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:15:20.862 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:15:20.880 INFO:tasks.cephadm:Teardown complete 2024-05-08T22:15:20.880 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T22:15:20.914 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T22:15:20.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:15:20.927 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:15:20.931 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:15:21.035 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T22:15:21.215 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T22:15:21.216 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:15:21.219 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:15:21.221 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:15:21.237 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-05-08T22:15:21.237 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2024-05-08T22:15:21.237 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2024-05-08T22:15:21.241 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:15:21.241 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-08T22:15:21.241 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +1706us[+1684us] +/- 58ms 2024-05-08T22:15:21.241 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1325us[-1347us] +/- 79ms 2024-05-08T22:15:21.241 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -667us[ -667us] +/- 79ms 2024-05-08T22:15:21.242 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +1664us[+1692us] +/- 58ms 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -1422us[-1395us] +/- 79ms 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -731us[ -704us] +/- 79ms 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:15:21.244 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:15:21.245 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-08T22:15:21.245 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +1682us[+1677us] +/- 58ms 2024-05-08T22:15:21.245 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -1353us[-1353us] +/- 79ms 2024-05-08T22:15:21.245 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -709us[ -709us] +/- 79ms 2024-05-08T22:15:21.245 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:15:21.245 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T22:15:21.256 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T22:15:21.256 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T22:15:21.321 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:15:21.402 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:15:21.473 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:15:21.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:15:21.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:15:21.573 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:15:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:15:21.605 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:15:21.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:15:21.635 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T22:15:21.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T22:15:21.667 INFO:teuthology.task.internal:Duration was 2587.004512 seconds 2024-05-08T22:15:21.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T22:15:21.683 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T22:15:21.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:15:21.686 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:15:21.688 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:15:21.734 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:15:21.734 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:15:21.734 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:15:22.154 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T22:15:22.154 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-05-08T22:15:22.155 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:15:22.180 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-05-08T22:15:22.180 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:15:22.203 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-05-08T22:15:22.204 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:15:22.226 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T22:15:22.226 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:15:22.229 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:15:22.246 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:15:22.318 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T22:15:22.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:15:22.320 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:15:22.322 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:15:22.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T22:15:22.442 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T22:15:22.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:15:22.470 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:15:22.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:15:22.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T22:15:22.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:15:22.537 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:15:22.539 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:15:22.562 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-05-08T22:15:22.564 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-05-08T22:15:22.590 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-05-08T22:15:22.620 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:15:22.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:15:22.637 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:15:22.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:15:22.653 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:15:22.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:15:22.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T22:15:22.688 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T22:15:22.690 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi064 2024-05-08T22:15:22.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:15:22.734 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi160 2024-05-08T22:15:22.734 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:15:22.775 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698067/remote/smithi186 2024-05-08T22:15:22.776 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:15:22.816 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T22:15:22.817 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:15:22.819 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:15:22.821 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:15:22.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T22:15:22.881 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T22:15:22.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T22:15:22.891 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T22:15:22.892 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:15:22.894 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:15:22.896 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:15:22.909 INFO:teuthology.orchestra.run.smithi064.stdout: 262158 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:15 /home/ubuntu/cephtest 2024-05-08T22:15:22.910 INFO:teuthology.orchestra.run.smithi160.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:15 /home/ubuntu/cephtest 2024-05-08T22:15:22.924 INFO:teuthology.orchestra.run.smithi186.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:15 /home/ubuntu/cephtest 2024-05-08T22:15:22.925 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T22:15:22.937 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T22:15:22.962 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 2587.004512310028 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T22:15:22.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:15:23.048 INFO:teuthology.run:pass